-- Logs begin at Thu 2022-07-07 09:43:04 UTC, end at Thu 2022-07-07 10:44:03 UTC. -- Jul 07 10:11:53.183062 dv-cinder-822821-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 07 10:11:53.191981 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jul 7 10:11:53 2022] *** Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: nodename: dv-cinder-822821-4-iscsi Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: machine: x86_64 Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: clock source: unix Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: pcre jit disabled Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: detected number of CPU cores: 4 Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: current working directory: / Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: detected binary path: /usr/bin/uwsgi-core Jul 07 10:11:53.194542 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: your processes number limit is 47680 Jul 07 10:11:53.195141 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: your memory page size is 4096 bytes Jul 07 10:11:53.195141 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: detected max file descriptor number: 2048 Jul 07 10:11:53.195141 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: lock engine: pthread robust mutexes Jul 07 10:11:53.195141 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: thunder lock: enabled Jul 07 10:11:53.195141 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 07 10:11:53.195141 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 07 10:11:53.238446 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: Python main interpreter initialized at 0x55cee86739c0 Jul 07 10:11:53.238446 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: python threads support enabled Jul 07 10:11:53.238446 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: your server socket listen backlog is limited to 100 connections Jul 07 10:11:53.238446 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: your mercy for graceful operations on workers is 90 seconds Jul 07 10:11:53.238762 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: mapped 403077 bytes (393 KB) for 2 cores Jul 07 10:11:53.238858 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: *** Operational MODE: preforking *** Jul 07 10:11:53.239125 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: *** uWSGI is running in multiple interpreter mode *** Jul 07 10:11:53.239125 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: spawned uWSGI master process (pid: 80132) Jul 07 10:11:53.239326 dv-cinder-822821-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 07 10:11:53.239902 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: spawned uWSGI worker 1 (pid: 80141, cores: 1) Jul 07 10:11:53.240675 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: spawned uWSGI worker 2 (pid: 80142, cores: 1) Jul 07 10:11:53.240675 dv-cinder-822821-4-iscsi devstack@keystone.service[80132]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 07 10:11:58.659795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 10:11:58.659795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Jul 07 10:11:58.660565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 10:11:58.660565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: from cryptography.utils import int_from_bytes Jul 07 10:11:58.660704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 10:11:58.660704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 07 10:11:58.661134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 07 10:11:58.661134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: from cryptography.utils import int_from_bytes Jul 07 10:11:58.709632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 07 10:11:58.709843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 07 10:11:58.710037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] command line args: [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 07 10:11:58.710195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 07 10:11:58.710304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 07 10:11:58.710415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 07 10:11:58.710502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ================================================================================ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 07 10:11:58.710636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] command line args: [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 07 10:11:58.710712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.710932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 07 10:11:58.711008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_dir = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.711123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.711209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ================================================================================ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 07 10:11:58.711307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] config_source = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.711405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.711480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] debug = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.711668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_dir = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] config_source = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] debug = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.712994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_config_append = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_dir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.713894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.714059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_config_append = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.714237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.714416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.714593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.714737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_dir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.714904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.715081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.715253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.715437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.715610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.715828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.715976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.716157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.716336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.716519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.716702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.716870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.717047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.717215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.717392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.717567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.717739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.717914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.718083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.718230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.718409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.718563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.718705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.718860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.718860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] publish_errors = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.719852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.720204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.16/identity {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.720204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] publish_errors = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.720428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.720542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.720653 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] standard_threads = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.720784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.720908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_journal = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_json = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.721905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] standard_threads = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_stderr = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] use_syslog = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.722797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_journal = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.722912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.723027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_json = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.723027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_stderr = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.723287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.723399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] use_syslog = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.723523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.723643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 07 10:11:58.723774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.723932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.724937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.725064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.725666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.725783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.725894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.726950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.727981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.728889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.729944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730864 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.730977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.731910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.732939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.733917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.734938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735415 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.735955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.736955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.737879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.738934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.739903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.740973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.741610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.741728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.741841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.741954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.742885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.743961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.744900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.745891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.746946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.747932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.external = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.748990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.password = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] auth.token = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.external = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.749964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.password = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] auth.token = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.750903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.751974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.752907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.753961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.754976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.755190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.755304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.755427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.755550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.755664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.755819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.755936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.756998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757806 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.757921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.758924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.759975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.760910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.761963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.762999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.763987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.764910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.765998 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.766999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.767962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.768896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.769920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.770970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.771086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.771200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.771315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.771429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.771552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.771668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.771957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.772566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.772686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.772801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.772928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773394 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.773976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.774901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.775965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.776900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.777949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.778990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.779979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.780905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781473 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.781932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.782933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.783910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.784894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.785919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.786957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.787073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.787187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.787187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.787400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.787514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.787636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.788342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.788467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.788583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.788698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.788812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.788941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.789991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.790929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.791977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.792904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.793957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.794907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.795978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.796915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.797967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80141) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.798763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.799691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.799691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 07 10:11:58.799691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80142) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 07 10:11:58.825471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.825703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.825934 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.826889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.827128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.827359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.835047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.836287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.864105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.864336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.866539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:58.866777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.006017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.006847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.054839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.055027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.056019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.056256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.120157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.120389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.120618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.120820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.120926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.121050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.121132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.121268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.121353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.121483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.121698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.121911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.146987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.147633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.147792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.147876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.147950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.148087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.158829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.159257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.172980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.173113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.173197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.173325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80141) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.173409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.173541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.173758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80142) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 07 10:11:59.190625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.190832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 10:11:59.191416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.191504 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.191615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 10:11:59.191711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 10:11:59.191910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.192053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.192169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.192251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 07 10:11:59.192420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.192505 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.192669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.192750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.193003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.193261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.204153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.204425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.204687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.204769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.204960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.205057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.205234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.205517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.213801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.214369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.217699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.217978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.218261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.218347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.218540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.218623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.218818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.218902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.219976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.220163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.220256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.220427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.220508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.220696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.220962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.247017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.248667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.248950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.251484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.253123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.253401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.254261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.254543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.254824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.259180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.259519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.259948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.262811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.263090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.267598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.267913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.268201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.268487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.268769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.269044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.269314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.269581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.273921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.274393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.282581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.283077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.283534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.284039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.284499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.284958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.285442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.285924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.289001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.289286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.293890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.297550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.297845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.303011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.306737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.306832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.307283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.310379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.310684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.310980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.311275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.311818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.311919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.313919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.317816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.318108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.323761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.327347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.330993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.331289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.331571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.331770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.331905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.332058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.332169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.332327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.332444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.332590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.341114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.342833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.343105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.346402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.346688 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.346974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.347260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.347631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.353264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.353547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.353808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.354067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.361017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.361300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.361584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.361868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.361951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.362151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.362440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.362731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.363647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.363954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.367894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.368172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.368449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.368726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.369021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.382604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.382729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.383005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.383282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.383571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.383876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.385533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.385821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.385904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.389223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.390846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.392631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.392911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.393195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.401146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.402763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.404867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.406016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.406309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.406600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.406881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.408159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.411437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.413230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.415492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.415936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.416077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.416162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.416396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.416679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.423371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.424565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80142) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.424848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.425004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.425129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.425403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.425674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.425952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.426239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.426514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.426889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.427160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.427432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.427695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.427997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.428286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80142) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.428464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.428750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.429139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.429416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.439438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.439723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.440027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.448661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80141) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 07 10:11:59.448939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.449217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.449490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.449757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.450139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.450414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.450689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.450964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.451231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.451498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.451778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.452054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.452315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80141) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 07 10:11:59.467507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cee86739c0 pid: 80142 (default app) Jul 07 10:11:59.470939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6bccc-eb5d-41f8-b0f9-a93b74bedfe6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:11:59.471245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6bccc-eb5d-41f8-b0f9-a93b74bedfe6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:11:59.471531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6bccc-eb5d-41f8-b0f9-a93b74bedfe6 None None] PATH_INFO: `/v3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:11:59.476404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1/1] 10.222.0.16 () {50 vars in 791 bytes} [Thu Jul 7 10:11:59 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:11:59.492454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cee86739c0 pid: 80141 (default app) Jul 07 10:12:07.060945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5a32d-ea52-4223-8b73-8ee9fc0f35dc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:07.061512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5a32d-ea52-4223-8b73-8ee9fc0f35dc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:07.061512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c5a32d-ea52-4223-8b73-8ee9fc0f35dc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:07.062850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 2/2] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:07.070747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:07.071176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:07.071305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:07.140072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: INFO dbcounter [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] Registered counter for database keystone Jul 07 10:12:07.161452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG oslo_db.sqlalchemy.engines [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80141) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 07 10:12:07.168227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writer thread running {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 07 10:12:07.341422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80141) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 07 10:12:07.341836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG passlib.handlers.bcrypt [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80141) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 07 10:12:07.395137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:07.397300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:07.517042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1/3] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:07 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 450 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 07 10:12:07.721190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5a32d-ea52-4223-8b73-8ee9fc0f35dc None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:07.797209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: INFO dbcounter [None req-c5c5a32d-ea52-4223-8b73-8ee9fc0f35dc None None] Registered counter for database keystone Jul 07 10:12:07.816827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG oslo_db.sqlalchemy.engines [None req-c5c5a32d-ea52-4223-8b73-8ee9fc0f35dc None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80142) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 07 10:12:07.823656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writer thread running {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 07 10:12:07.940454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c5a32d-ea52-4223-8b73-8ee9fc0f35dc None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:07.945504 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6f3a86-bfb5-4fb9-bd02-e47de8ecf2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:07.946166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f3a86-bfb5-4fb9-bd02-e47de8ecf2bc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:07.946456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f3a86-bfb5-4fb9-bd02-e47de8ecf2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:07.946737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6f3a86-bfb5-4fb9-bd02-e47de8ecf2bc admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:07.954771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f3a86-bfb5-4fb9-bd02-e47de8ecf2bc admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:08.005480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2a6f3a86-bfb5-4fb9-bd02-e47de8ecf2bc admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 07 10:12:08.005826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6f3a86-bfb5-4fb9-bd02-e47de8ecf2bc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:08.015242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 3/4] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Jul 7 10:12:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:12:08.021191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:08.036298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27948051-3d15-43bc-a5a2-d94efb9b8c12 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:08.041337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271afa42-116e-42ea-adf2-aae8992398bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:08.042454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271afa42-116e-42ea-adf2-aae8992398bb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:08.042743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271afa42-116e-42ea-adf2-aae8992398bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:08.043026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271afa42-116e-42ea-adf2-aae8992398bb admin admin] PATH_INFO: `/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:08.047103 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271afa42-116e-42ea-adf2-aae8992398bb admin admin] RBAC: Authorizing `identity:get_project(project_id=3e93b5f74d6546a8b33441fdeb0e9c0d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:08.097217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING keystone.common.rbac_enforcer.enforcer [None req-271afa42-116e-42ea-adf2-aae8992398bb admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 07 10:12:08.097751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271afa42-116e-42ea-adf2-aae8992398bb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:08.102691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 2/5] 10.222.0.16 () {56 vars in 1207 bytes} [Thu Jul 7 10:12:08 2022] GET /identity/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:10.078445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec6290-b85e-43f8-a0ee-efd842beaa4a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:10.078978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec6290-b85e-43f8-a0ee-efd842beaa4a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:10.079073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ec6290-b85e-43f8-a0ee-efd842beaa4a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:10.083183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 4/6] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:10.088644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495ea99-6227-4dbe-8fb6-43d23b427ee4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:10.089058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495ea99-6227-4dbe-8fb6-43d23b427ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:10.089201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c495ea99-6227-4dbe-8fb6-43d23b427ee4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:10.107833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c495ea99-6227-4dbe-8fb6-43d23b427ee4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:10.109343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c495ea99-6227-4dbe-8fb6-43d23b427ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:10.117966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 3/7] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:10 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 07 10:12:10.140351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec6290-b85e-43f8-a0ee-efd842beaa4a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:10.152351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ec6290-b85e-43f8-a0ee-efd842beaa4a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:10.156556 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b72cd1-5b2b-457d-8c92-9fa09f61731b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:10.157188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b72cd1-5b2b-457d-8c92-9fa09f61731b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:10.157480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b72cd1-5b2b-457d-8c92-9fa09f61731b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:10.157763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b72cd1-5b2b-457d-8c92-9fa09f61731b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:10.165458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b72cd1-5b2b-457d-8c92-9fa09f61731b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:10.167251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b72cd1-5b2b-457d-8c92-9fa09f61731b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:10.176058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 5/8] 10.222.0.16 () {58 vars in 1324 bytes} [Thu Jul 7 10:12:10 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:12:10.181549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495ea99-6227-4dbe-8fb6-43d23b427ee4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:10.190362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c495ea99-6227-4dbe-8fb6-43d23b427ee4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:10.195491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1412b0da-e07b-4470-8d4c-2a2434385ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:10.196369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412b0da-e07b-4470-8d4c-2a2434385ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:10.196654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412b0da-e07b-4470-8d4c-2a2434385ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:10.196931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1412b0da-e07b-4470-8d4c-2a2434385ebe admin admin] PATH_INFO: `/v3/users/8d3febf927af4782bc2aa5631cc4dde7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:10.198311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412b0da-e07b-4470-8d4c-2a2434385ebe admin admin] RBAC: Authorizing `identity:get_user(user_id=8d3febf927af4782bc2aa5631cc4dde7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:10.200164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1412b0da-e07b-4470-8d4c-2a2434385ebe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:10.201779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 4/9] 10.222.0.16 () {56 vars in 1198 bytes} [Thu Jul 7 10:12:10 2022] GET /identity/v3/users/8d3febf927af4782bc2aa5631cc4dde7 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:12.270366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88164d-be48-4928-9635-7a0834d1469b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.270908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88164d-be48-4928-9635-7a0834d1469b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.271051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f88164d-be48-4928-9635-7a0834d1469b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.272385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 6/10] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:12.277547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eba997-f044-4b61-9b44-46ac416bc3a6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.278029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eba997-f044-4b61-9b44-46ac416bc3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.278125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eba997-f044-4b61-9b44-46ac416bc3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.279772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52fb3c-a7f2-4ce9-9fed-bf1ffe2268d0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.280072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52fb3c-a7f2-4ce9-9fed-bf1ffe2268d0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.280354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee52fb3c-a7f2-4ce9-9fed-bf1ffe2268d0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.281622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 7/11] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:12.286478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.286801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.287088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.294486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a6eba997-f044-4b61-9b44-46ac416bc3a6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:12.295700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a6eba997-f044-4b61-9b44-46ac416bc3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:12.327375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80142) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 07 10:12:12.327836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG passlib.handlers.bcrypt [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80142) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 07 10:12:12.373063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:12.374670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:12.376137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 5/12] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:12.399531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eba997-f044-4b61-9b44-46ac416bc3a6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:12.423262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:12.476086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 8/13] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:12.497313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:12.504137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eba997-f044-4b61-9b44-46ac416bc3a6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:12.540436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:12.541178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.541471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.541747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.553684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:12.565027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:12.566328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:12.567151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 6/14] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:12:12.572859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:12.596917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0a0adf-4f67-4a89-984b-252ed512815a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:12.599190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:12.638533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:12.639438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.639731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.640030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.653075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:12.673046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958e6a8-b058-4b48-b165-c301c7b4e88a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:12.706992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:12.707156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:12.707657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.707967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.708248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.708742 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:12.709045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:12.709356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:12.710737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:12.722225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 7/15] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:12.728251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:12.804576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 9/16] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:12:12.809946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:12.812571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:12.825784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df010c16-7735-414c-86d6-9039a9ed5950 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:12.859414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:12.860317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.860605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.860882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] PATH_INFO: `/v3/users/8d3febf927af4782bc2aa5631cc4dde7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.862096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] RBAC: Authorizing `identity:get_user(user_id=8d3febf927af4782bc2aa5631cc4dde7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:12.863590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:12.865797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 8/17] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:12 2022] GET /identity/v3/users/8d3febf927af4782bc2aa5631cc4dde7 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:12.871036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:12.885212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:12.907052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df191a-b000-4024-b834-29c8ab631632 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:12.941524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:12.942319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:12.942606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:12.942911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:12.960800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:12.962286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:12.969579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8628134-5de5-4b08-b507-dbcf69b5414d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:12:12.985558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:12.986392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:12.987165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:12:12.987165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:12:12.987165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:12:12.987165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:12.987165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 10/18] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:12:12.993240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:13.008849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:13.009714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:13.010003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:13.010278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:13.011449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:13.012945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:13.014520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 9/19] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:13.020157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:13.049226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:13.092576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2088b7-a79d-4c81-b5aa-1b54f26fc4c2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:13.104733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:13.124195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f95819-de66-4444-8b15-7d3f1fa4f92a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:13.125480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec529bac-e9d3-41c5-b0fa-2eb20dd1024a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:13.126152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec529bac-e9d3-41c5-b0fa-2eb20dd1024a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:13.126446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec529bac-e9d3-41c5-b0fa-2eb20dd1024a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:13.126729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec529bac-e9d3-41c5-b0fa-2eb20dd1024a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:13.127464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec529bac-e9d3-41c5-b0fa-2eb20dd1024a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:13.127802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec529bac-e9d3-41c5-b0fa-2eb20dd1024a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:13.129199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec529bac-e9d3-41c5-b0fa-2eb20dd1024a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:13.146647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 11/20] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:12:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:13.157661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296c5faf-72c8-4b94-aabe-0c3e24126057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:13.158347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c5faf-72c8-4b94-aabe-0c3e24126057 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:13.158634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c5faf-72c8-4b94-aabe-0c3e24126057 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:13.158915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296c5faf-72c8-4b94-aabe-0c3e24126057 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:13.159788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c5faf-72c8-4b94-aabe-0c3e24126057 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.domain.id=default, user.id=8d3febf927af4782bc2aa5631cc4dde7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:13.160110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c5faf-72c8-4b94-aabe-0c3e24126057 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:13.162164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296c5faf-72c8-4b94-aabe-0c3e24126057 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:13.169843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 10/21] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Jul 7 10:12:13 2022] GET /identity/v3/role_assignments?user.id=8d3febf927af4782bc2aa5631cc4dde7&scope.domain.id=default&role.id=f5019e69971843dfadbbc6be956053fb => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:15.529861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232cec-84fc-4f45-a127-14a940c8bae1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:15.530332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232cec-84fc-4f45-a127-14a940c8bae1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:15.530404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07232cec-84fc-4f45-a127-14a940c8bae1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:15.531967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 12/22] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:15.537365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d071-5f01-476d-b50b-e70b91208976 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:15.537768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d071-5f01-476d-b50b-e70b91208976 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:15.537918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c7d071-5f01-476d-b50b-e70b91208976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:15.564914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-47c7d071-5f01-476d-b50b-e70b91208976 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:15.566598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47c7d071-5f01-476d-b50b-e70b91208976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:15.637427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 11/23] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:15.671967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07232cec-84fc-4f45-a127-14a940c8bae1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:15.823575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07232cec-84fc-4f45-a127-14a940c8bae1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:15.845045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:15.866817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e349ee-1c6a-4986-9c85-94543084d415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:15.867630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e349ee-1c6a-4986-9c85-94543084d415 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:15.867940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e349ee-1c6a-4986-9c85-94543084d415 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:15.868218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e349ee-1c6a-4986-9c85-94543084d415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:15.885203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e349ee-1c6a-4986-9c85-94543084d415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b6e349ee-1c6a-4986-9c85-94543084d415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:15.894128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:15.894931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:15.895671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 13/24] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:15.903513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7d071-5f01-476d-b50b-e70b91208976 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:15.937572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c74718-eee8-4bdb-8593-e61ddb28e37f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:15.937866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c74718-eee8-4bdb-8593-e61ddb28e37f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:15.938132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c74718-eee8-4bdb-8593-e61ddb28e37f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:15.939809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 14/25] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:15.943350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808d6a-3ce6-45b7-b7c5-91169bafb0f9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:15.943639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808d6a-3ce6-45b7-b7c5-91169bafb0f9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:15.943947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11808d6a-3ce6-45b7-b7c5-91169bafb0f9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:15.945445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 15/26] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:15.946612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c5182-b95a-4722-9350-1017f15702ef None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:15.946902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c5182-b95a-4722-9350-1017f15702ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:15.947179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854c5182-b95a-4722-9350-1017f15702ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:15.977032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-854c5182-b95a-4722-9350-1017f15702ef None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:15.978233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-854c5182-b95a-4722-9350-1017f15702ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:16.034423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:16.072522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c7d071-5f01-476d-b50b-e70b91208976 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:16.082602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 16/27] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:16.084433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8e87-df46-4a60-a699-179631fce45a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.084867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8e87-df46-4a60-a699-179631fce45a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.085287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed8e87-df46-4a60-a699-179631fce45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.102508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c8ed8e87-df46-4a60-a699-179631fce45a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:16.104019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c8ed8e87-df46-4a60-a699-179631fce45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:16.119356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762c64af-7777-4b8b-9164-7f90ef30b542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:16.120066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c64af-7777-4b8b-9164-7f90ef30b542 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.120523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c64af-7777-4b8b-9164-7f90ef30b542 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.120939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762c64af-7777-4b8b-9164-7f90ef30b542 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.121827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c64af-7777-4b8b-9164-7f90ef30b542 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:16.122283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c64af-7777-4b8b-9164-7f90ef30b542 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:16.126015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762c64af-7777-4b8b-9164-7f90ef30b542 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:16.134712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 12/28] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:16.136126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38d17-5b8f-44a7-a8a7-ae72c88e75f2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.136552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38d17-5b8f-44a7-a8a7-ae72c88e75f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.136989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c38d17-5b8f-44a7-a8a7-ae72c88e75f2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.138645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 13/29] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:16.143418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c38d17-5b8f-44a7-a8a7-ae72c88e75f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:16.177326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 17/30] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:16.178245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed8e87-df46-4a60-a699-179631fce45a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:16.215413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:16.269806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c38d17-5b8f-44a7-a8a7-ae72c88e75f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:16.288764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed8e87-df46-4a60-a699-179631fce45a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:16.307085 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:16.308501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.309078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.309586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.310521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:16.312181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:16.318259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:16.329648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8042d-8a10-470c-a598-de65b2f875b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:16.330872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8042d-8a10-470c-a598-de65b2f875b6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.331464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8042d-8a10-470c-a598-de65b2f875b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.332036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8042d-8a10-470c-a598-de65b2f875b6 None admin] PATH_INFO: `/v3/users/8d3febf927af4782bc2aa5631cc4dde7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.334201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8042d-8a10-470c-a598-de65b2f875b6 None admin] RBAC: Authorizing `identity:get_user(user_id=8d3febf927af4782bc2aa5631cc4dde7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:16.335885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8042d-8a10-470c-a598-de65b2f875b6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:16.337731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 18/31] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:16 2022] GET /identity/v3/users/8d3febf927af4782bc2aa5631cc4dde7 => generated 247 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:16.339118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d4788-47e7-43f9-abee-bedc48df83b9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.339542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d4788-47e7-43f9-abee-bedc48df83b9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.340004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d4788-47e7-43f9-abee-bedc48df83b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.358271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-287d4788-47e7-43f9-abee-bedc48df83b9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:16.359878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-287d4788-47e7-43f9-abee-bedc48df83b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:16.435156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 19/32] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:16.436658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d4788-47e7-43f9-abee-bedc48df83b9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:16.472781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 14/33] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jul 7 10:12:16 2022] POST /identity/v3/roles => generated 225 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:16.474287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:16.496222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:16.580602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d4788-47e7-43f9-abee-bedc48df83b9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:16.643143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e70129-6bd5-4125-97ce-2fe58c3f2feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:16.644111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e70129-6bd5-4125-97ce-2fe58c3f2feb None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.644561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e70129-6bd5-4125-97ce-2fe58c3f2feb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.644976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e70129-6bd5-4125-97ce-2fe58c3f2feb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.646596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e70129-6bd5-4125-97ce-2fe58c3f2feb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:16.648061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e70129-6bd5-4125-97ce-2fe58c3f2feb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:16.661478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f3f09c-45ac-48f1-981d-2c5ca7ac514b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:16.699278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:16.700373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.701395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.701890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.703565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:16.705527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:16.707365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 15/34] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:16.708094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:16.771871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:16.776034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 20/35] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jul 7 10:12:16 2022] POST /identity/v3/roles => generated 219 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:16.777584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc0ff9-5a46-4cbd-8d83-5fa4a520081d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.777881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc0ff9-5a46-4cbd-8d83-5fa4a520081d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.778164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc0ff9-5a46-4cbd-8d83-5fa4a520081d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.779484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 21/36] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:16.780294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc0ff9-5a46-4cbd-8d83-5fa4a520081d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:16.833845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bdd9bc-dbae-4a3d-a2a3-d1a775229203 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:16.873507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24075e8-0ecb-4ef8-9528-3206fefd8a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:16.874333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24075e8-0ecb-4ef8-9528-3206fefd8a5b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.874620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24075e8-0ecb-4ef8-9528-3206fefd8a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.875491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24075e8-0ecb-4ef8-9528-3206fefd8a5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.887652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24075e8-0ecb-4ef8-9528-3206fefd8a5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:16.889252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24075e8-0ecb-4ef8-9528-3206fefd8a5b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:16.898685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:16.913190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc0ff9-5a46-4cbd-8d83-5fa4a520081d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c24075e8-0ecb-4ef8-9528-3206fefd8a5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:12:16.914349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:16.915136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 16/37] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f98a0-ed73-4c3e-a9ed-e0a5ad152098 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f98a0-ed73-4c3e-a9ed-e0a5ad152098 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.920893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f98a0-ed73-4c3e-a9ed-e0a5ad152098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.938326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5e7f98a0-ed73-4c3e-a9ed-e0a5ad152098 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:16.940019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5e7f98a0-ed73-4c3e-a9ed-e0a5ad152098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:16.962344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:16.963702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:16.964228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:16.964682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] PATH_INFO: `/v3/domains/default/users/8d3febf927af4782bc2aa5631cc4dde7/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:16.967128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=8d3febf927af4782bc2aa5631cc4dde7, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:16.968731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:16.992620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 22/38] 10.222.0.16 () {58 vars in 1361 bytes} [Thu Jul 7 10:12:16 2022] PUT /identity/v3/domains/default/users/8d3febf927af4782bc2aa5631cc4dde7/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:16.998126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:17.031566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 17/39] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:17.066579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f98a0-ed73-4c3e-a9ed-e0a5ad152098 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:17.130691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:17.147785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f09fc3-7230-4e9d-a3e0-1bda603cbf8f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:17.195669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f98a0-ed73-4c3e-a9ed-e0a5ad152098 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:17.197283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:17.198566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:17.198566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:17.198566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:17.199277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:17.199587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:17.201112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:17.213347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 23/40] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:12:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:17.218670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:17.237322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ecdab9-5824-4cd3-a020-d27338749af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:17.238005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecdab9-5824-4cd3-a020-d27338749af3 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:17.238293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecdab9-5824-4cd3-a020-d27338749af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:17.238577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecdab9-5824-4cd3-a020-d27338749af3 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:17.239328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecdab9-5824-4cd3-a020-d27338749af3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:17.241348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ecdab9-5824-4cd3-a020-d27338749af3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:17.247442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:17.259349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 18/41] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jul 7 10:12:17 2022] POST /identity/v3/roles => generated 223 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:12:17.318375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f4b08-3341-4d37-8781-eab47a1dc99e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:17.319118 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f4b08-3341-4d37-8781-eab47a1dc99e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:17.319652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9f4b08-3341-4d37-8781-eab47a1dc99e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:17.321747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 19/42] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:17.327977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279c397-f9f9-4671-8aa5-8c331e6dda56 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:17.328662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279c397-f9f9-4671-8aa5-8c331e6dda56 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:17.329182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b279c397-f9f9-4671-8aa5-8c331e6dda56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:17.349275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b279c397-f9f9-4671-8aa5-8c331e6dda56 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:17.351028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b279c397-f9f9-4671-8aa5-8c331e6dda56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:17.357862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820f873-38c9-4625-b239-c4b77b0b9c38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:17.408461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:17.409695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:17.410004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:17.410291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:17.411033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:17.414020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:17.435052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 20/43] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:17.444250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 24/44] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:12:17 2022] POST /identity/v3/projects => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:12:17.458825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279c397-f9f9-4671-8aa5-8c331e6dda56 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:17.618113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:17.684055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b279c397-f9f9-4671-8aa5-8c331e6dda56 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:17.718284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22b97a6-4347-4f4b-8f30-3ad6c4a1082c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:17.719016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b97a6-4347-4f4b-8f30-3ad6c4a1082c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:17.719293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b97a6-4347-4f4b-8f30-3ad6c4a1082c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:17.719564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22b97a6-4347-4f4b-8f30-3ad6c4a1082c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:17.720759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b97a6-4347-4f4b-8f30-3ad6c4a1082c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:17.722530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22b97a6-4347-4f4b-8f30-3ad6c4a1082c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:17.724801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 21/45] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:17.730562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:17.794031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:17.840342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b514293a-e4a9-4ab7-aea5-d665c3ccd162 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:17.872613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bf8099-7f40-4cab-a17d-af4453434772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:17.873278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8099-7f40-4cab-a17d-af4453434772 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:17.873571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8099-7f40-4cab-a17d-af4453434772 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:17.873851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bf8099-7f40-4cab-a17d-af4453434772 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:17.874576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8099-7f40-4cab-a17d-af4453434772 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:17.876064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bf8099-7f40-4cab-a17d-af4453434772 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:17.900681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 25/46] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:12:17 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:19.091499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb72bb9-1496-4ac2-8b03-ee92eda02f6b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.092044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb72bb9-1496-4ac2-8b03-ee92eda02f6b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.092142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb72bb9-1496-4ac2-8b03-ee92eda02f6b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.093470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 22/47] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:19.098754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0088-cdca-470d-a44a-b01645014a48 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.099183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0088-cdca-470d-a44a-b01645014a48 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.099448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd0088-cdca-470d-a44a-b01645014a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.116573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-93dd0088-cdca-470d-a44a-b01645014a48 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:19.118094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-93dd0088-cdca-470d-a44a-b01645014a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:19.183282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 26/48] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:19.206749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb72bb9-1496-4ac2-8b03-ee92eda02f6b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:19.306386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb72bb9-1496-4ac2-8b03-ee92eda02f6b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:19.308669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:19.340453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:19.341254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.341587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.341815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.351858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:19.359513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:19.360385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:19.361374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 23/49] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:19.367059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0088-cdca-470d-a44a-b01645014a48 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:19.414580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:19.465083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dd0088-cdca-470d-a44a-b01645014a48 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:19.498325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:19.499004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.499294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.499578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.500371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:19.500686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:19.502080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:19.515016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 27/50] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:19.520620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:19.619050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba707ab-50cd-4c12-b0bf-6cbe63d33cdd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:19.651333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:19.652236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.652530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.652808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] PATH_INFO: `/v3/users/8d3febf927af4782bc2aa5631cc4dde7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.653992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=8d3febf927af4782bc2aa5631cc4dde7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:19.655433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:19.657099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 24/51] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:19 2022] GET /identity/v3/users/8d3febf927af4782bc2aa5631cc4dde7 => generated 247 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:19.662969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:19.867122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cb149-78e1-41d5-8024-b85963389397 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:19.877154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:19.900840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f32087-7123-4133-8213-195f1b8256df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:19.901649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f32087-7123-4133-8213-195f1b8256df None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.901938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f32087-7123-4133-8213-195f1b8256df None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.902220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f32087-7123-4133-8213-195f1b8256df None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.903400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f32087-7123-4133-8213-195f1b8256df None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:19.904894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f32087-7123-4133-8213-195f1b8256df None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:19.906518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 28/52] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:19.912986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:19.937795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d144-b256-4a3e-8b7a-9c8f5898fae5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.938092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d144-b256-4a3e-8b7a-9c8f5898fae5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.938405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c5d144-b256-4a3e-8b7a-9c8f5898fae5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.939931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 29/53] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:19.944003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:19.944865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d9baf-9aef-450c-a985-e48818c4d29b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:19.945164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d9baf-9aef-450c-a985-e48818c4d29b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:19.945484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d9baf-9aef-450c-a985-e48818c4d29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:19.962386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1b0d9baf-9aef-450c-a985-e48818c4d29b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:19.963600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1b0d9baf-9aef-450c-a985-e48818c4d29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:20.012570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c5faa-89bf-4818-95c3-80b323121cc3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:20.029202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 30/54] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:20.045950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:20.046602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:20.046892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:20.047174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:20.048032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.domain.id=default, user.id=8d3febf927af4782bc2aa5631cc4dde7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:20.048350 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:20.049807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:20.052947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d9baf-9aef-450c-a985-e48818c4d29b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:20.055615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 25/55] 10.222.0.16 () {56 vars in 1322 bytes} [Thu Jul 7 10:12:19 2022] GET /identity/v3/role_assignments?user.id=8d3febf927af4782bc2aa5631cc4dde7&scope.domain.id=default&role.id=f5019e69971843dfadbbc6be956053fb => generated 530 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:20.157047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d9baf-9aef-450c-a985-e48818c4d29b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:20.167633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:20.193276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35eb655-8282-4865-b1b7-83892a38b817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:20.194083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eb655-8282-4865-b1b7-83892a38b817 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:20.194365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eb655-8282-4865-b1b7-83892a38b817 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:20.194638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35eb655-8282-4865-b1b7-83892a38b817 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:20.195867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35eb655-8282-4865-b1b7-83892a38b817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:20.197310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35eb655-8282-4865-b1b7-83892a38b817 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:20.198928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 31/56] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:20.204655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:20.300838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:20.307111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8a1a47-23e3-475d-9cc1-d693626447a4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:20.340486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc63f368-452a-48d9-a6ed-099ef6e9d25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:20.341148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f368-452a-48d9-a6ed-099ef6e9d25c None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:20.341436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f368-452a-48d9-a6ed-099ef6e9d25c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:20.341715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc63f368-452a-48d9-a6ed-099ef6e9d25c None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:20.342422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63f368-452a-48d9-a6ed-099ef6e9d25c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:20.343908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc63f368-452a-48d9-a6ed-099ef6e9d25c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:20.369840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 26/57] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Jul 7 10:12:20 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:22.376995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de9dc-c4d1-4301-949f-ca828b4bb63e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:22.377538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de9dc-c4d1-4301-949f-ca828b4bb63e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:22.377538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55de9dc-c4d1-4301-949f-ca828b4bb63e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:22.378920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 32/58] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:22.384006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72e380-5672-42b4-be4e-ea51eb013792 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:22.384407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72e380-5672-42b4-be4e-ea51eb013792 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:22.384580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72e380-5672-42b4-be4e-ea51eb013792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:22.401419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8f72e380-5672-42b4-be4e-ea51eb013792 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:22.402613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8f72e380-5672-42b4-be4e-ea51eb013792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:22.468588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 27/59] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:22.492429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55de9dc-c4d1-4301-949f-ca828b4bb63e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:22.589901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55de9dc-c4d1-4301-949f-ca828b4bb63e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:22.621476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab4c1e0-d9b3-4043-b8cc-aca988100672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:22.622267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4c1e0-d9b3-4043-b8cc-aca988100672 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:22.622553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4c1e0-d9b3-4043-b8cc-aca988100672 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:22.622831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab4c1e0-d9b3-4043-b8cc-aca988100672 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:22.624023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4c1e0-d9b3-4043-b8cc-aca988100672 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:22.625457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab4c1e0-d9b3-4043-b8cc-aca988100672 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:22.627090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 33/60] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:22.632972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72e380-5672-42b4-be4e-ea51eb013792 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:22.708641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:22.730446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72e380-5672-42b4-be4e-ea51eb013792 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:22.763001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cc2f23-82d9-4d9d-a82b-25492d13784c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:22.763664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc2f23-82d9-4d9d-a82b-25492d13784c None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:22.763993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc2f23-82d9-4d9d-a82b-25492d13784c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:22.764278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cc2f23-82d9-4d9d-a82b-25492d13784c None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:22.764985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc2f23-82d9-4d9d-a82b-25492d13784c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:22.766447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cc2f23-82d9-4d9d-a82b-25492d13784c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:22.790422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 28/61] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:12:22 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:25.853115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f23da-842d-468e-9ed6-670daf6616ad None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:25.854140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f23da-842d-468e-9ed6-670daf6616ad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:25.854889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542f23da-842d-468e-9ed6-670daf6616ad None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:25.856516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 34/62] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:25.862028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa91b3-72da-4ef0-8be6-e8aa0840aa31 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:25.862464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa91b3-72da-4ef0-8be6-e8aa0840aa31 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:25.862597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fa91b3-72da-4ef0-8be6-e8aa0840aa31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:25.881071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d0fa91b3-72da-4ef0-8be6-e8aa0840aa31 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:25.882297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d0fa91b3-72da-4ef0-8be6-e8aa0840aa31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:25.960928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 29/63] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:25.996939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f23da-842d-468e-9ed6-670daf6616ad None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.067546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:26.114725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71828-b3e3-49bd-8070-b1e8ab829627 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.115344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71828-b3e3-49bd-8070-b1e8ab829627 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.115864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e71828-b3e3-49bd-8070-b1e8ab829627 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.117457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 30/64] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:26.122908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261282-ca37-42ea-80c5-f93e0efb3c44 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.123434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261282-ca37-42ea-80c5-f93e0efb3c44 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.123985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19261282-ca37-42ea-80c5-f93e0efb3c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.142788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-19261282-ca37-42ea-80c5-f93e0efb3c44 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:26.144373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-19261282-ca37-42ea-80c5-f93e0efb3c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:26.146360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542f23da-842d-468e-9ed6-670daf6616ad None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.184678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20bb38f-29f8-417b-8b4a-3880963bbd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.185753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bb38f-29f8-417b-8b4a-3880963bbd2d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.186178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bb38f-29f8-417b-8b4a-3880963bbd2d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.186601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20bb38f-29f8-417b-8b4a-3880963bbd2d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.198348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20bb38f-29f8-417b-8b4a-3880963bbd2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f20bb38f-29f8-417b-8b4a-3880963bbd2d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:26.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:26.207243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:26.208074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:26.210736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 35/65] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:26.212279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a17a9-aac9-4cf2-919d-cec195d8a857 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.212745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a17a9-aac9-4cf2-919d-cec195d8a857 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.213194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a17a9-aac9-4cf2-919d-cec195d8a857 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.214627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 36/66] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:26.219620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a17a9-aac9-4cf2-919d-cec195d8a857 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.221967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:26.234642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 31/67] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:26.236670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ebd3d-cad7-48a7-a9fe-0de00aa2d2bf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.237162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ebd3d-cad7-48a7-a9fe-0de00aa2d2bf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.237598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8ebd3d-cad7-48a7-a9fe-0de00aa2d2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.266454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1b8ebd3d-cad7-48a7-a9fe-0de00aa2d2bf None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:26.267981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1b8ebd3d-cad7-48a7-a9fe-0de00aa2d2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:26.349763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 32/68] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:26.350991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ebd3d-cad7-48a7-a9fe-0de00aa2d2bf None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.371941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a17a9-aac9-4cf2-919d-cec195d8a857 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.375320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:26.408948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.409656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.409955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.410227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.411510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:26.411848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:26.413216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:26.425648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 37/69] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:26.425648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.477693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8ebd3d-cad7-48a7-a9fe-0de00aa2d2bf None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.501090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:26.518022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab858f71-7497-4861-95f2-55a6b9d44d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.518858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab858f71-7497-4861-95f2-55a6b9d44d35 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.519189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab858f71-7497-4861-95f2-55a6b9d44d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.519463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab858f71-7497-4861-95f2-55a6b9d44d35 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.531364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab858f71-7497-4861-95f2-55a6b9d44d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:26.538460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66896b9-9621-49dd-8c12-9acb35b34097 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ab858f71-7497-4861-95f2-55a6b9d44d35 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:26.540034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:26.540869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:26.541639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 33/70] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:26.543514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f17b8-680d-4fd6-8b35-16476b691635 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.543514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f17b8-680d-4fd6-8b35-16476b691635 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.543514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35f17b8-680d-4fd6-8b35-16476b691635 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.548209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 34/71] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:26 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:26.548209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f17b8-680d-4fd6-8b35-16476b691635 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.578439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.579287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.579583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.579909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.591368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:26.599944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:26.600766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:26.601530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 38/72] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:26.602160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.631669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:26.653983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35f17b8-680d-4fd6-8b35-16476b691635 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.687309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cd903f-8522-45eb-b53a-55292f5768f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.688225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd903f-8522-45eb-b53a-55292f5768f0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.688514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd903f-8522-45eb-b53a-55292f5768f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.688798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cd903f-8522-45eb-b53a-55292f5768f0 None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.702208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd903f-8522-45eb-b53a-55292f5768f0 None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:26.703690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cd903f-8522-45eb-b53a-55292f5768f0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:26.703810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad45fa9b-2b51-4528-95fa-19bd7f9282f6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.705481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 35/73] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:26.706662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb41296-bbaf-4d81-8252-ce4f5814b058 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.706954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb41296-bbaf-4d81-8252-ce4f5814b058 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.707239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb41296-bbaf-4d81-8252-ce4f5814b058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.723867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fbb41296-bbaf-4d81-8252-ce4f5814b058 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:26.725064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fbb41296-bbaf-4d81-8252-ce4f5814b058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:26.744882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df280b92-1dce-477a-9470-612bfc4c4fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.745566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280b92-1dce-477a-9470-612bfc4c4fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.745860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280b92-1dce-477a-9470-612bfc4c4fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.746143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df280b92-1dce-477a-9470-612bfc4c4fb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.746900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df280b92-1dce-477a-9470-612bfc4c4fb0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:26.747212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df280b92-1dce-477a-9470-612bfc4c4fb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:26.748618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df280b92-1dce-477a-9470-612bfc4c4fb0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:26.756374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 39/74] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:26.757713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30042d-03bf-43ec-abce-292815d9c668 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.758014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30042d-03bf-43ec-abce-292815d9c668 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.758298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df30042d-03bf-43ec-abce-292815d9c668 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.759666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 40/75] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:26.760154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:26.764325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30042d-03bf-43ec-abce-292815d9c668 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.793488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 36/76] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:26.794318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb41296-bbaf-4d81-8252-ce4f5814b058 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.854526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:26.867771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df30042d-03bf-43ec-abce-292815d9c668 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.893508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb41296-bbaf-4d81-8252-ce4f5814b058 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:26.901307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.901981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.902286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.902545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.903302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:26.903614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:26.905027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:26.913869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 41/77] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:26.914651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:26.931876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:26.932540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:26.932831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:26.933109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:26.974154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.000597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.002266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.012830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bec3c3d-73c7-4764-b3b6-a5b07cdde6dd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.046562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0218b-b57b-4278-90de-37e088f906b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.047440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0218b-b57b-4278-90de-37e088f906b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.047731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0218b-b57b-4278-90de-37e088f906b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.048043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0218b-b57b-4278-90de-37e088f906b3 None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.049272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0218b-b57b-4278-90de-37e088f906b3 None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.050707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0218b-b57b-4278-90de-37e088f906b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.052578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 42/78] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:27.053539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7923132-5b3a-4394-a666-3c9ed338335c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.053832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7923132-5b3a-4394-a666-3c9ed338335c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.054119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7923132-5b3a-4394-a666-3c9ed338335c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.070377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d7923132-5b3a-4394-a666-3c9ed338335c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:27.071576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d7923132-5b3a-4394-a666-3c9ed338335c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:27.086333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.105456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 37/79] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:27.106293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.142700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 43/80] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:27.143653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7923132-5b3a-4394-a666-3c9ed338335c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.175710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.204872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1536b43-ce6a-4130-b489-130c8f179dbe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.238296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.239157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.239441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.239715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.242851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7923132-5b3a-4394-a666-3c9ed338335c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.250222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:27.258625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:27.259428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 38/81] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:27.260639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.280565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.281453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.281742 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.282017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] PATH_INFO: `/v3/users/8d3febf927af4782bc2aa5631cc4dde7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.283233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] RBAC: Authorizing `identity:get_user(user_id=8d3febf927af4782bc2aa5631cc4dde7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.284724 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.286372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 44/82] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/users/8d3febf927af4782bc2aa5631cc4dde7 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:27.287126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.307917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.362242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b5a25a-3ca3-4c9e-9688-0be9b4dd67c9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.388491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8be6de-b5bf-4803-8779-78cb0397533b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.391331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.399654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.400269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.400558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.400834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.422505 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.423375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.423666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.423982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.425193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.427146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.429096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 45/83] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:27.429644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.472662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.474185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.518831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.535464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39c1632-3dfa-4218-9424-c5823370043e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.568882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.569681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.569969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.570251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.571420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.572883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.574541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 46/84] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:27.575293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.580234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 39/85] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:27.580234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.607837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.674297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d07afb-bc0a-48fe-8fed-ae68ed4fe957 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.675084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946531e-8921-4e5f-821e-dae3d34a11e3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.708241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.708429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.708913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.709003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.709204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.709316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.709489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.709593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.710243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:27.710553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.711952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.719838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 47/86] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:27.720622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.736076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.777791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.779279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.807583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.818439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfce3db9-4bd9-4b80-8deb-84558d144c76 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:27.851017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:27.851668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:27.851983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:27.852265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:27.853096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=9930e234fb15434b881890a315de561e, user.id=646457ebc2974d8dad3b9b7800cf8ac7) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:27.853410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:27.854840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:27.870482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 48/87] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/role_assignments?user.id=646457ebc2974d8dad3b9b7800cf8ac7&scope.project.id=9930e234fb15434b881890a315de561e&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:27.871279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.876447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 40/88] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:27.877329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:27.981355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:27.991352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edb849e-252d-4546-9a8e-50f4b4d7b09e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.005749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aad2a9-64e0-4335-b4a2-8475e3139a72 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.027250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:28.032190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.033066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.033357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.033642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.034949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.036448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.042777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 41/89] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:27 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:28.042777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:28.054856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.055905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.056207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.056650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.057400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.058866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.092391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 49/90] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:12:27 2022] POST /identity/v3/users => generated 289 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:28.092391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:28.162029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193d3ba-7f66-4ef6-bd41-463e2abab4c8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.202766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.203811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.204105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.204383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.205775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.207312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.209561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 42/91] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:28 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:28.210531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:28.245774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80193d66-ee57-4ebc-8ea3-ff80d89e9ac2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.289421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.290629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.291092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.291381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.293620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.294531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.296002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:28.296113 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 50/92] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:28 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:28.296862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:28.300193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:28.339152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30760e48-ebfd-47d4-ba1e-92189360a1fe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.380301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.380968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.381251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.381522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.382360 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] RBAC: Adding query filter params (role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=9930e234fb15434b881890a315de561e, user.id=646457ebc2974d8dad3b9b7800cf8ac7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:28.382681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.385352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.391291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 43/93] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:28 2022] GET /identity/v3/role_assignments?user.id=646457ebc2974d8dad3b9b7800cf8ac7&scope.project.id=9930e234fb15434b881890a315de561e&role.id=7673131020174664baa4d10a6f38ce2e => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:28.391966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:28.445578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05a7527-cf56-416a-8c7c-d1e0a6cf490f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.490755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.491443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.491754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.492059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.542964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3ea7cb-4dc8-4196-bd00-396daeedc08b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.585119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.586850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:28.588063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.590273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:28.596109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.597084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.597571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.598077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.599119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=9930e234fb15434b881890a315de561e, user.id=8d3febf927af4782bc2aa5631cc4dde7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:28.600242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.601945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.610478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 44/94] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:28 2022] GET /identity/v3/role_assignments?user.id=8d3febf927af4782bc2aa5631cc4dde7&scope.project.id=9930e234fb15434b881890a315de561e&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:28.738284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 51/95] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:28.745138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:28.871250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac8bad-0ff9-43c1-8662-b068db2ee5d0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:28.908166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf9f0b-e8a0-4a29-92eb-b94df63f2435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:28.909405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf9f0b-e8a0-4a29-92eb-b94df63f2435 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:28.909892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf9f0b-e8a0-4a29-92eb-b94df63f2435 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:28.910986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf9f0b-e8a0-4a29-92eb-b94df63f2435 None admin] PATH_INFO: `/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:28.912467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf9f0b-e8a0-4a29-92eb-b94df63f2435 None admin] RBAC: Authorizing `identity:get_project(project_id=1e067025a1ba4c6c8b3f38213caf81c8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:28.914142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf9f0b-e8a0-4a29-92eb-b94df63f2435 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:28.916092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 45/96] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:28 2022] GET /identity/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8 => generated 312 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:28.921944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:29.071576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fc2f67-c369-4a15-b60c-80f28a15f2c5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:29.073776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:29.125456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60fca1a-be69-4c60-9bc3-6562d3397945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:29.126464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fca1a-be69-4c60-9bc3-6562d3397945 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:29.126970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fca1a-be69-4c60-9bc3-6562d3397945 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:29.127446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60fca1a-be69-4c60-9bc3-6562d3397945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:29.128535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fca1a-be69-4c60-9bc3-6562d3397945 None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=1e067025a1ba4c6c8b3f38213caf81c8, user.id=646457ebc2974d8dad3b9b7800cf8ac7) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:29.129086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fca1a-be69-4c60-9bc3-6562d3397945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:29.130815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60fca1a-be69-4c60-9bc3-6562d3397945 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:29.139454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 52/97] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:28 2022] GET /identity/v3/role_assignments?user.id=646457ebc2974d8dad3b9b7800cf8ac7&scope.project.id=1e067025a1ba4c6c8b3f38213caf81c8&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:30.489915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa693b0-c273-49f2-9fb8-2403a5200979 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:30.490423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa693b0-c273-49f2-9fb8-2403a5200979 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:30.490498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa693b0-c273-49f2-9fb8-2403a5200979 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:30.492154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 46/98] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:30.497170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042b3c-113d-48ac-a811-ec290afb8530 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:30.497982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042b3c-113d-48ac-a811-ec290afb8530 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:30.498482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042b3c-113d-48ac-a811-ec290afb8530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:30.522267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ad042b3c-113d-48ac-a811-ec290afb8530 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:30.523532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ad042b3c-113d-48ac-a811-ec290afb8530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:30.608383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 53/99] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:30.642962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa693b0-c273-49f2-9fb8-2403a5200979 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:30.727328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:30.777493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa693b0-c273-49f2-9fb8-2403a5200979 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:30.821730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:30.822551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:30.822841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:30.823128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:30.834450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:30.846374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:30.847209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:30.848011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 47/100] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:30.854825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042b3c-113d-48ac-a811-ec290afb8530 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:30.994256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:31.006394 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad042b3c-113d-48ac-a811-ec290afb8530 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:31.046778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3087ae0-04f4-44d9-b50a-a2b82c6b401e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:31.047790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3087ae0-04f4-44d9-b50a-a2b82c6b401e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.048367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3087ae0-04f4-44d9-b50a-a2b82c6b401e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.048835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3087ae0-04f4-44d9-b50a-a2b82c6b401e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.049781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3087ae0-04f4-44d9-b50a-a2b82c6b401e None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:31.050282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3087ae0-04f4-44d9-b50a-a2b82c6b401e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:31.052474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3087ae0-04f4-44d9-b50a-a2b82c6b401e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:31.067615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 54/101] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:31.073341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:31.085997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919d1ba-b6dd-4df4-aff5-a0b6e4ed3f82 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.086289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919d1ba-b6dd-4df4-aff5-a0b6e4ed3f82 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.086577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e919d1ba-b6dd-4df4-aff5-a0b6e4ed3f82 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.088129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 55/102] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:31.092931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dfad0-0ab1-4021-a85a-f1d845bf4ead None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.093221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dfad0-0ab1-4021-a85a-f1d845bf4ead None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.093501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12dfad0-0ab1-4021-a85a-f1d845bf4ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.119005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e12dfad0-0ab1-4021-a85a-f1d845bf4ead None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:31.120250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e12dfad0-0ab1-4021-a85a-f1d845bf4ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:31.240691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 56/103] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:31.241907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e04feb-8bc7-4d75-99c8-c397c6bc5e94 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.242207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e04feb-8bc7-4d75-99c8-c397c6bc5e94 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.242493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e04feb-8bc7-4d75-99c8-c397c6bc5e94 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.243931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 57/104] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:31.249302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6a8ea-148e-45ef-ab83-cd7c6263a583 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.249603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6a8ea-148e-45ef-ab83-cd7c6263a583 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.249890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f6a8ea-148e-45ef-ab83-cd7c6263a583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.267940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-29f6a8ea-148e-45ef-ab83-cd7c6263a583 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:31.269351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29f6a8ea-148e-45ef-ab83-cd7c6263a583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:31.269944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546535db-ee73-4450-bd38-b12878bf6178 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:31.321963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:31.341419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:31.342309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.342594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.342872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.345036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:31.346524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:31.351387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 48/105] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:31 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:31.352191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:31.386318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 58/106] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:31.388451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6a8ea-148e-45ef-ab83-cd7c6263a583 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:31.418044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:31.527621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61516f35-b8a6-4959-8934-893cc1d8dc99 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:31.587134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f6a8ea-148e-45ef-ab83-cd7c6263a583 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:31.597891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:31.598699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.598982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.599259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.615694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:31.623765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:31.624766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:31.625618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 49/107] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:31.626340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:31.688159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:31.701059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:31.701971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.702264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.702552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.703831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:31.705315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:31.707968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 59/108] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:31 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:31.708809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:31.732902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44207080-9296-45cd-89c9-d986af5569b2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:31.770613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:31.771419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.771705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.772024 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.773209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:31.774680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:31.782978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 50/109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:31.783850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:31.842365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:31.858977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea450041-cdc8-48ce-8259-d56270fd9d34 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:31.893211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:31.893896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.894192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.895000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.895799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:31.896131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:31.897507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:31.903619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c739bd6d-49cf-4c7f-b607-3ae127612999 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:31.907187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 60/110] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:31.908125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:31.929429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:31.944971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e160f3-87d3-4a57-b42c-27baebb429e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:31.946026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e160f3-87d3-4a57-b42c-27baebb429e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.946318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e160f3-87d3-4a57-b42c-27baebb429e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.947369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e160f3-87d3-4a57-b42c-27baebb429e0 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e/users/646457ebc2974d8dad3b9b7800cf8ac7/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.949390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e160f3-87d3-4a57-b42c-27baebb429e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=9930e234fb15434b881890a315de561e, user_id=646457ebc2974d8dad3b9b7800cf8ac7, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:31.950845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e160f3-87d3-4a57-b42c-27baebb429e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:31.974783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 51/111] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:31 2022] PUT /identity/v3/projects/9930e234fb15434b881890a315de561e/users/646457ebc2974d8dad3b9b7800cf8ac7/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:31.974783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed9db3-718a-49b9-a7f2-5aaa203956b2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.974783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed9db3-718a-49b9-a7f2-5aaa203956b2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.974783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed9db3-718a-49b9-a7f2-5aaa203956b2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.974783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 52/112] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:31.974783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852bf08-c7e3-4e3a-be1f-741feb69b410 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:31.975341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852bf08-c7e3-4e3a-be1f-741feb69b410 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:31.975341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852bf08-c7e3-4e3a-be1f-741feb69b410 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:31.975341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 53/113] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:31.975341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852bf08-c7e3-4e3a-be1f-741feb69b410 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:32.039541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f902a-61ae-450f-a2c7-133cf5d3b4c7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:32.085833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd06e71b-17c0-4ad7-aa6c-31e5f1f93cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:32.086760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06e71b-17c0-4ad7-aa6c-31e5f1f93cda None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.087193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06e71b-17c0-4ad7-aa6c-31e5f1f93cda None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.087644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd06e71b-17c0-4ad7-aa6c-31e5f1f93cda None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.088661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06e71b-17c0-4ad7-aa6c-31e5f1f93cda None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:32.090425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd06e71b-17c0-4ad7-aa6c-31e5f1f93cda None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:32.096575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852bf08-c7e3-4e3a-be1f-741feb69b410 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:32.117937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:32.136116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 61/114] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:12:31 2022] POST /identity/v3/projects => generated 302 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:32.136116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d8f9d-cb94-4255-9552-33735881943c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.136116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d8f9d-cb94-4255-9552-33735881943c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.136116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83d8f9d-cb94-4255-9552-33735881943c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.153745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357d0d31-2a49-47a5-b187-d60ce892040f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:32.154699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d0d31-2a49-47a5-b187-d60ce892040f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.154990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d0d31-2a49-47a5-b187-d60ce892040f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.155295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357d0d31-2a49-47a5-b187-d60ce892040f None admin] PATH_INFO: `/v3/users/df7c8088fc764cf4899956ca28fc4463` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.162083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a83d8f9d-cb94-4255-9552-33735881943c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:32.163619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a83d8f9d-cb94-4255-9552-33735881943c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:32.174986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d0d31-2a49-47a5-b187-d60ce892040f None admin] RBAC: Authorizing `identity:get_user(user_id=df7c8088fc764cf4899956ca28fc4463)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:32.176647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357d0d31-2a49-47a5-b187-d60ce892040f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:32.178584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 54/115] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:31 2022] GET /identity/v3/users/df7c8088fc764cf4899956ca28fc4463 => generated 289 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:32.180007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e69a6-6ae0-49ed-b1eb-cca2357ebe72 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.180394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e69a6-6ae0-49ed-b1eb-cca2357ebe72 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.181179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48e69a6-6ae0-49ed-b1eb-cca2357ebe72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.201245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e48e69a6-6ae0-49ed-b1eb-cca2357ebe72 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:32.202507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e48e69a6-6ae0-49ed-b1eb-cca2357ebe72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:32.241744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:32.265340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 62/116] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:32.266503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d8f9d-cb94-4255-9552-33735881943c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:32.309834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 55/117] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:32.311310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958e634-6ec7-4f07-a934-3eeb8ef43f57 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.311674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958e634-6ec7-4f07-a934-3eeb8ef43f57 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.312000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958e634-6ec7-4f07-a934-3eeb8ef43f57 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.313424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 56/118] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:32.314092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958e634-6ec7-4f07-a934-3eeb8ef43f57 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:32.395880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83d8f9d-cb94-4255-9552-33735881943c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:32.432128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:32.440635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:32.442114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.442446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.443375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.460550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e958e634-6ec7-4f07-a934-3eeb8ef43f57 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:32.510307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6ce293-a7a2-4b8c-8960-f839ad3686b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:32.511160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ce293-a7a2-4b8c-8960-f839ad3686b5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.511447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ce293-a7a2-4b8c-8960-f839ad3686b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.511729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6ce293-a7a2-4b8c-8960-f839ad3686b5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.526487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6ce293-a7a2-4b8c-8960-f839ad3686b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:32.528996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:32.530490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0c6ce293-a7a2-4b8c-8960-f839ad3686b5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:32.534543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:32.535776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:32.536594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 57/119] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:32.540987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47f9a0-28c9-4f9d-929d-b7c907628ff9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.541282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47f9a0-28c9-4f9d-929d-b7c907628ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.541566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47f9a0-28c9-4f9d-929d-b7c907628ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.560656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-df47f9a0-28c9-4f9d-929d-b7c907628ff9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:32.561933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-df47f9a0-28c9-4f9d-929d-b7c907628ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:32.587558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:32.639014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 63/120] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:32.639898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:32.643036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 58/121] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:32.644942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47f9a0-28c9-4f9d-929d-b7c907628ff9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:32.709678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:32.768589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47f9a0-28c9-4f9d-929d-b7c907628ff9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:32.784485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5ad19-1faa-4d0c-bf5c-ed01c666e33d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:32.819323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:32.820527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.821381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.822133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.823203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:32.823844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:32.825771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:32.826100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:32.827023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:32.827368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:32.827660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:32.837555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 59/122] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:32.838620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:32.844036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:32.852721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:32.853557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:32.854333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 64/123] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:12:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:32.854949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:32.876862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:32.973650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:32.975700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6b2ca-c81c-449c-9507-2d2763885b62 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:32.986824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85864280-4e60-492d-96e1-1979736cdab2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.012859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.013761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.014049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.014346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.015607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.017491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.023927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 60/124] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:32 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.023927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.036319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.037339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.037939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.038382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.052192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:33.062410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:33.063268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:33.064063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 65/125] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:33.068929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.122763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:33.137417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4ef895-6032-46e6-88dc-95e07d02a955 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.173925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.174798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.175088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.175371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] PATH_INFO: `/v3/users/8d3febf927af4782bc2aa5631cc4dde7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.177310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] RBAC: Authorizing `identity:get_user(user_id=8d3febf927af4782bc2aa5631cc4dde7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.178930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.180635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 61/126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/users/8d3febf927af4782bc2aa5631cc4dde7 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.185955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.195395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7144a077-2aca-422e-8029-9ef7888a1273 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.235508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.236176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.236470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.236762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.237533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:33.237852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.239990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.249261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 66/127] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.253801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.270230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:33.334812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153ecb14-a48f-4297-b89a-ec45dfffbfb1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.370920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.372020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.373011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.373483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.374512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] RBAC: Adding query filter params (role.id=70a666d4a1c5476790ac8f65c8a167f3, scope.project.id=9930e234fb15434b881890a315de561e, user.id=df7c8088fc764cf4899956ca28fc4463) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:33.375137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.376918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.382841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 62/128] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/role_assignments?user.id=df7c8088fc764cf4899956ca28fc4463&scope.project.id=9930e234fb15434b881890a315de561e&role.id=70a666d4a1c5476790ac8f65c8a167f3 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.384020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.416234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:33.549889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8ee976-783a-4ae6-b0a5-ddc41b3414af None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.599799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786fb056-9fe7-42b6-80cb-12f88f0d94ad None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.622553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.624648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.625874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.626408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.628104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:33.628905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.631174 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.651446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 67/129] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.658649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.675768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.675768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.675768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.675768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.676157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.676157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.676705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 63/130] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.677536 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.830406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:33.842716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdea17b-8751-4343-a77c-04e71860106b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.848737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:33.900557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3789e9-956a-430a-9c67-c72e3917330d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:33.904442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.905345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.905635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.905914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.912513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.914018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.915774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 64/131] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.916530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:33.960468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:33.961383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:33.961683 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:33.961972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:33.963247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:33.964735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:33.967375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 68/132] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:33.968204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:34.020518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28e4ea6-3e15-46e7-8be4-0730277a5b12 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:34.057150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:34.058049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.058335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.058614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e/users/8d3febf927af4782bc2aa5631cc4dde7/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.060796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] RBAC: Authorizing `identity:create_grant(project_id=9930e234fb15434b881890a315de561e, user_id=8d3febf927af4782bc2aa5631cc4dde7, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:34.062258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:34.078185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 65/133] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:33 2022] PUT /identity/v3/projects/9930e234fb15434b881890a315de561e/users/8d3febf927af4782bc2aa5631cc4dde7/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:34.079070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:34.124245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:34.172136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e090a9-7401-4556-8bad-5a2ded6b96aa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:34.240408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:34.241508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.241901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.242284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] PATH_INFO: `/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.244937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] RBAC: Authorizing `identity:get_project(project_id=1e067025a1ba4c6c8b3f38213caf81c8)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:34.246516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:34.248469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 69/134] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:33 2022] GET /identity/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8 => generated 312 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:34.257059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:34.278031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:34.306538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5309cde3-322b-4fb3-b9b1-195e3910d58b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:34.369733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:34.370863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.371308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.371729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.373137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:34.374798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:34.376692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 66/135] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:34 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:34.382573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:34.425066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0e1930-7418-4e52-ad43-7ff4ecbc5c55 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:34.462364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3900d-4f9e-42ab-ac29-923eff2af414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:34.463288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3900d-4f9e-42ab-ac29-923eff2af414 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.463584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3900d-4f9e-42ab-ac29-923eff2af414 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.463905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3900d-4f9e-42ab-ac29-923eff2af414 None admin] PATH_INFO: `/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8/users/646457ebc2974d8dad3b9b7800cf8ac7/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.465979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3900d-4f9e-42ab-ac29-923eff2af414 None admin] RBAC: Authorizing `identity:create_grant(project_id=1e067025a1ba4c6c8b3f38213caf81c8, user_id=646457ebc2974d8dad3b9b7800cf8ac7, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:34.467554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd3900d-4f9e-42ab-ac29-923eff2af414 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:34.484869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 70/136] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:34 2022] PUT /identity/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8/users/646457ebc2974d8dad3b9b7800cf8ac7/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:12:34.489787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:34.518767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6e6ed-30a7-4ed3-8b21-2ad8b900d0b6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.519137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6e6ed-30a7-4ed3-8b21-2ad8b900d0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.523964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad6e6ed-30a7-4ed3-8b21-2ad8b900d0b6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.525502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 71/137] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:34 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:34.526203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14602445-bb43-43b7-8aa8-9ce343d81ad6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:34.537904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff58e3-1ff9-4355-a56e-8813b8b64498 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.537904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff58e3-1ff9-4355-a56e-8813b8b64498 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.537904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eff58e3-1ff9-4355-a56e-8813b8b64498 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.569599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3eff58e3-1ff9-4355-a56e-8813b8b64498 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:34.569789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b8961b-4aaf-411a-889c-08cf48707b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:34.570551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8961b-4aaf-411a-889c-08cf48707b92 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.570838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8961b-4aaf-411a-889c-08cf48707b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.570935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3eff58e3-1ff9-4355-a56e-8813b8b64498 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:34.571116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b8961b-4aaf-411a-889c-08cf48707b92 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e/users/646457ebc2974d8dad3b9b7800cf8ac7/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.573614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8961b-4aaf-411a-889c-08cf48707b92 None admin] RBAC: Authorizing `identity:create_grant(project_id=9930e234fb15434b881890a315de561e, user_id=646457ebc2974d8dad3b9b7800cf8ac7, role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:34.575287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b8961b-4aaf-411a-889c-08cf48707b92 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:34.599248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 67/138] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:34 2022] PUT /identity/v3/projects/9930e234fb15434b881890a315de561e/users/646457ebc2974d8dad3b9b7800cf8ac7/roles/7673131020174664baa4d10a6f38ce2e => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:34.605057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683efc4-badf-4ecb-be1d-d5b7b2e68af0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.605684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683efc4-badf-4ecb-be1d-d5b7b2e68af0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.605684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3683efc4-badf-4ecb-be1d-d5b7b2e68af0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.607788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 68/139] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:34.621678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7d4dc-d0c6-46ff-a3eb-0c21d1c1c73b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.622238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7d4dc-d0c6-46ff-a3eb-0c21d1c1c73b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.622683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df7d4dc-d0c6-46ff-a3eb-0c21d1c1c73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.657674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4df7d4dc-d0c6-46ff-a3eb-0c21d1c1c73b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:34.659540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4df7d4dc-d0c6-46ff-a3eb-0c21d1c1c73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:34.713435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:34.720234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 72/140] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:34.765575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff58e3-1ff9-4355-a56e-8813b8b64498 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:34.772502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 69/141] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:34.801421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7d4dc-d0c6-46ff-a3eb-0c21d1c1c73b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:34.885898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eff58e3-1ff9-4355-a56e-8813b8b64498 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:34.931426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44853783-c83a-498e-a793-91c068e41490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:34.932381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44853783-c83a-498e-a793-91c068e41490 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:34.932805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44853783-c83a-498e-a793-91c068e41490 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:34.933355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44853783-c83a-498e-a793-91c068e41490 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:34.938280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:34.954928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44853783-c83a-498e-a793-91c068e41490 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-44853783-c83a-498e-a793-91c068e41490 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:34.966627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:34.967713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:34.968510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:34.969079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 73/142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:34.975191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44853783-c83a-498e-a793-91c068e41490 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:34.977139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df7d4dc-d0c6-46ff-a3eb-0c21d1c1c73b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:35.022909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:35.023770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:35.024053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:35.024331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:35.025511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:35.026992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:35.029027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 70/143] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:35.034095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:35.056620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:35.103806 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44853783-c83a-498e-a793-91c068e41490 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:35.143599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:35.144773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:35.145212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:35.145577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:35.146387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:35.146760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:35.148272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:35.157216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 74/144] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:35.163395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:35.170036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab13a6dd-2b94-4611-9145-1f49e54eea3e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:35.211887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:35.212950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:35.213437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:35.213983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:35.214883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:35.216818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:35.242638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:35.251926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 71/145] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:12:35 2022] POST /identity/v3/users => generated 283 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:12:35.289548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2333a67e-e2da-4a2d-8383-81965c06b2f2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:35.315410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:35.336271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:35.337530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:35.338018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:35.338596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:35.340101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:35.341812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:35.343681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 75/146] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:35 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:35.349788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:35.488561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a54d0-8659-4c9d-9518-d060eb664f1e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:35.526534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:35.527201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:35.527488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:35.527783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:35.603072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:35.606890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:35.608431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:35.715398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 72/147] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:35.721225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:35.830589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e346672-ac4e-458d-af7e-61219404bc27 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:35.867006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d792a-7b7d-4dbd-b593-57af3663de3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:35.867915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d792a-7b7d-4dbd-b593-57af3663de3a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:35.868208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d792a-7b7d-4dbd-b593-57af3663de3a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:35.868492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d792a-7b7d-4dbd-b593-57af3663de3a None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:35.869701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d792a-7b7d-4dbd-b593-57af3663de3a None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:35.871165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d792a-7b7d-4dbd-b593-57af3663de3a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:35.872912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 76/148] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:35 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:35.879605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:35.986951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:35.991340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcb20c8-0bc4-4ce6-aaeb-83ee108e5fe4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:36.026870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fcdfa5-9639-4d20-bfff-4994932a551e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:36.027535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcdfa5-9639-4d20-bfff-4994932a551e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:36.027846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcdfa5-9639-4d20-bfff-4994932a551e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:36.028130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fcdfa5-9639-4d20-bfff-4994932a551e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:36.029042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcdfa5-9639-4d20-bfff-4994932a551e None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=9930e234fb15434b881890a315de561e, user.id=646457ebc2974d8dad3b9b7800cf8ac7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:36.029358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcdfa5-9639-4d20-bfff-4994932a551e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:36.030870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fcdfa5-9639-4d20-bfff-4994932a551e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:36.038037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 73/149] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:35 2022] GET /identity/v3/role_assignments?user.id=646457ebc2974d8dad3b9b7800cf8ac7&scope.project.id=9930e234fb15434b881890a315de561e&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:36.292019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7979c8c-2c19-4e3a-9298-787b3afa00d8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:36.292373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7979c8c-2c19-4e3a-9298-787b3afa00d8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:36.292544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7979c8c-2c19-4e3a-9298-787b3afa00d8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:36.294167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 77/150] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:36.299159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf652-c6b6-477b-bcd3-619008db231a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:36.299453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf652-c6b6-477b-bcd3-619008db231a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:36.299730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83cf652-c6b6-477b-bcd3-619008db231a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:36.323958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e83cf652-c6b6-477b-bcd3-619008db231a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:36.325219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e83cf652-c6b6-477b-bcd3-619008db231a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:36.411502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 74/151] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:36.438311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7979c8c-2c19-4e3a-9298-787b3afa00d8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:36.512081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:36.564770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7979c8c-2c19-4e3a-9298-787b3afa00d8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:36.609469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:36.610666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:36.611171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:36.611662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:36.632850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:36.643829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:36.646045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:36.647012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:36.649636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 78/152] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:36.658001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cf652-c6b6-477b-bcd3-619008db231a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:36.794543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83cf652-c6b6-477b-bcd3-619008db231a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:36.800215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:36.843376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:36.844012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:36.844298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:36.844578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:36.845531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:36.845824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:36.847397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:36.856998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 75/153] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:36.862669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:37.003300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ef428f-1cab-4699-ab1f-3c48a1ada0a2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:37.024508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:37.043076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:37.043984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:37.044266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:37.044541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] PATH_INFO: `/v3/users/df7c8088fc764cf4899956ca28fc4463` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:37.045749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] RBAC: Authorizing `identity:get_user(user_id=df7c8088fc764cf4899956ca28fc4463)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:37.047204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:37.048867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 79/154] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:36 2022] GET /identity/v3/users/df7c8088fc764cf4899956ca28fc4463 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:37.056559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:37.167287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2c4684-5bad-4c4c-be23-cf87dd29eca8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:37.207275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff84ebec-0aad-47dc-8def-c8e51c363198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:37.208182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84ebec-0aad-47dc-8def-c8e51c363198 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:37.208466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84ebec-0aad-47dc-8def-c8e51c363198 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:37.208747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff84ebec-0aad-47dc-8def-c8e51c363198 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:37.210116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84ebec-0aad-47dc-8def-c8e51c363198 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:37.211573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff84ebec-0aad-47dc-8def-c8e51c363198 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:37.213655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 76/155] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:37 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:37.218842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:37.352467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b56d0-20d9-423d-b0c9-a4be83ebd699 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:37.399913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61834634-813b-45f0-bf29-997225d03354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:37.400816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61834634-813b-45f0-bf29-997225d03354 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:37.401099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61834634-813b-45f0-bf29-997225d03354 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:37.401377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61834634-813b-45f0-bf29-997225d03354 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e/users/df7c8088fc764cf4899956ca28fc4463/roles/70a666d4a1c5476790ac8f65c8a167f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:37.403366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61834634-813b-45f0-bf29-997225d03354 None admin] RBAC: Authorizing `identity:create_grant(project_id=9930e234fb15434b881890a315de561e, user_id=df7c8088fc764cf4899956ca28fc4463, role_id=70a666d4a1c5476790ac8f65c8a167f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:37.408898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61834634-813b-45f0-bf29-997225d03354 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:37.427121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 80/156] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:37 2022] PUT /identity/v3/projects/9930e234fb15434b881890a315de561e/users/df7c8088fc764cf4899956ca28fc4463/roles/70a666d4a1c5476790ac8f65c8a167f3 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:37.702887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1dabe-b8da-4040-9191-51a742497bd4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:37.703378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1dabe-b8da-4040-9191-51a742497bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:37.703378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb1dabe-b8da-4040-9191-51a742497bd4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:37.705095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 77/157] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:37.717402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa0d9-0b76-462e-9576-8e6bf4784dbc None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:37.717720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa0d9-0b76-462e-9576-8e6bf4784dbc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:37.717974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795aa0d9-0b76-462e-9576-8e6bf4784dbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:37.742005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-795aa0d9-0b76-462e-9576-8e6bf4784dbc None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:37.743244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-795aa0d9-0b76-462e-9576-8e6bf4784dbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:37.796579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:37.830418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 81/158] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:37.847860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47bee8-274c-40ef-815f-038caac29aa1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:37.848159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47bee8-274c-40ef-815f-038caac29aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:37.848434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c47bee8-274c-40ef-815f-038caac29aa1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:37.849991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 78/159] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:37.855583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef249-10ce-43ec-8843-8b578f077618 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:37.856133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef249-10ce-43ec-8843-8b578f077618 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:37.856613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bef249-10ce-43ec-8843-8b578f077618 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:37.879217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-00bef249-10ce-43ec-8843-8b578f077618 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:37.881398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-00bef249-10ce-43ec-8843-8b578f077618 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:37.887142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47bee8-274c-40ef-815f-038caac29aa1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:37.930601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:38.018622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 82/160] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:38.053711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c47bee8-274c-40ef-815f-038caac29aa1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:38.057227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bef249-10ce-43ec-8843-8b578f077618 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:38.107787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:38.107787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:38.107787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:38.107787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:38.126374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:38.150945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:38.151940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:38.153182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 79/161] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:12:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:38.164175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:38.249248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bef249-10ce-43ec-8843-8b578f077618 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:38.262281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:38.305402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:38.313371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:38.314189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:38.314725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:38.315003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:38.332794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:38.342872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b3b321-6abc-43c8-be89-d7e91623ab0d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:38.344891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:38.345728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:38.346544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 83/162] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:38.354317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:38.409385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:38.410593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:38.410880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:38.411158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:38.411940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:38.412253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:38.413655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:38.422278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 80/163] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:12:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:38.428233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:38.514750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa40538-0af3-427b-b6f6-8055df4db3e4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:38.588859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06e52f4-70b2-4d4d-8bc7-8384e88942f9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:38.596455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:38.597218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:38.598023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:38.598303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:38.599066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:38.599379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:38.600788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:38.616551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 84/164] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:38.624750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:38.641367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:38.642264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:38.642548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:38.642829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:38.648467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:38.649954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:38.651631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 81/165] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:38 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:38.657571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:38.679368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:38.684904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:38.804271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4a8dc-a1be-4dd4-b941-0b714856885e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:38.828721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2331bdfd-a63f-4fa5-9f00-594589cf9e36 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:38.872525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:38.873342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:38.873627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:38.873905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] PATH_INFO: `/v3/users/8d3febf927af4782bc2aa5631cc4dde7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:38.875147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] RBAC: Authorizing `identity:get_user(user_id=8d3febf927af4782bc2aa5631cc4dde7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:38.876671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:38.878548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 85/166] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:38 2022] GET /identity/v3/users/8d3febf927af4782bc2aa5631cc4dde7 => generated 247 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:38.884185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:38.889410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9bc494-394c-4d98-82d1-4e3c59b670bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:38.890079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bc494-394c-4d98-82d1-4e3c59b670bb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:38.890368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bc494-394c-4d98-82d1-4e3c59b670bb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:38.890650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9bc494-394c-4d98-82d1-4e3c59b670bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:38.974998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bc494-394c-4d98-82d1-4e3c59b670bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:38.976529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9bc494-394c-4d98-82d1-4e3c59b670bb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:38.992098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:39.046410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:39.057486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d96c3c-7ec9-43a9-8152-0aeb845feedc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:39.083027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 82/167] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:12:39.084571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8f608-585a-4c46-a876-bdd283c00593 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.084894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8f608-585a-4c46-a876-bdd283c00593 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.085250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8f608-585a-4c46-a876-bdd283c00593 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.086680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 83/168] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:39.087702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b772f-b2c7-492d-85ad-eb988a6c3c3c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.088015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b772f-b2c7-492d-85ad-eb988a6c3c3c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.088387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b772f-b2c7-492d-85ad-eb988a6c3c3c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.089759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 84/169] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:39.095173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25b77-b9ea-4b98-9cbb-70fc2abea249 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.095461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25b77-b9ea-4b98-9cbb-70fc2abea249 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.095754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc25b77-b9ea-4b98-9cbb-70fc2abea249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.112526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4dc25b77-b9ea-4b98-9cbb-70fc2abea249 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:39.113758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4dc25b77-b9ea-4b98-9cbb-70fc2abea249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:39.125423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:39.126033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.126320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.126601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.182927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 85/170] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:39.196139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72d96-ca02-45fb-a5de-58c2ff4f46e8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.196440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72d96-ca02-45fb-a5de-58c2ff4f46e8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.198513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c72d96-ca02-45fb-a5de-58c2ff4f46e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.225155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a4c72d96-ca02-45fb-a5de-58c2ff4f46e8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:39.226541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a4c72d96-ca02-45fb-a5de-58c2ff4f46e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:39.252776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:39.255143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:39.264122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:39.310132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 86/171] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:39.310999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72d96-ca02-45fb-a5de-58c2ff4f46e8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:39.415073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c72d96-ca02-45fb-a5de-58c2ff4f46e8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:39.423217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:39.434335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 86/172] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:12:39.434963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:39.459035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:39.459936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.460223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.460503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.461801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:39.463262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:39.468679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 87/173] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:39 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:39.469450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:39.560111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:39.623163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59609068-036e-4f36-bf62-d9dc66f91f73 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:39.648774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08472b75-a73b-4dd0-bae9-13dc08e5e173 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:39.664179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:39.664179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.664179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.664179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.674116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:39.685222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:39.686206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 88/174] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:39.692348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:39.717986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:39.719156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.719711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.720209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.743498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:39.760852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:39.761785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:39.762582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:39.765472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 87/175] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:39.766568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:39.825053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:39.829983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3750d6d8-08d6-472f-9ff8-ab65794782b1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:39.873043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:39.896130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:39.897054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.897340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.897619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.901736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:39.903263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:39.905037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 89/176] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:39 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:39.905802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:39.950764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c58c25a-0977-4950-8799-df0a828a0e49 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:39.991730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:39.992943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:39.993451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:39.993948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:39.995195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] RBAC: Adding query filter params (role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=9930e234fb15434b881890a315de561e, user.id=646457ebc2974d8dad3b9b7800cf8ac7) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:39.995522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:39.997021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.007769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 88/177] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:39 2022] GET /identity/v3/role_assignments?user.id=646457ebc2974d8dad3b9b7800cf8ac7&scope.project.id=9930e234fb15434b881890a315de561e&role.id=7673131020174664baa4d10a6f38ce2e => generated 608 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:40.007769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.056765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e35058e-374c-478c-b66e-c5dd3673fcac None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:40.130532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:40.131234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.131516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.131825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.133209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:40.133531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:40.134951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.150629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 90/178] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:40.150629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.164238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03b76ac-5714-4898-81bd-cf770822b52a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:40.193319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:40.195251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:40.216401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ccfb5-2655-4caa-972b-5c80ccaa0a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:40.217407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ccfb5-2655-4caa-972b-5c80ccaa0a69 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.217407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ccfb5-2655-4caa-972b-5c80ccaa0a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.217869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ccfb5-2655-4caa-972b-5c80ccaa0a69 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.218733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ccfb5-2655-4caa-972b-5c80ccaa0a69 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:40.219937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ccfb5-2655-4caa-972b-5c80ccaa0a69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:40.220669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ccfb5-2655-4caa-972b-5c80ccaa0a69 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.231717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 89/179] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:40.233953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb524d-64d5-4cd6-aa41-31c6028444e1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.234338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb524d-64d5-4cd6-aa41-31c6028444e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.234713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb524d-64d5-4cd6-aa41-31c6028444e1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.236286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 90/180] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:40.237319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15166deb-3979-43db-a123-ddeabdd1d996 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.238210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15166deb-3979-43db-a123-ddeabdd1d996 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.238647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15166deb-3979-43db-a123-ddeabdd1d996 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.239963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 91/181] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:40.240765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15166deb-3979-43db-a123-ddeabdd1d996 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.293681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa3ca-7c6d-4890-b0d9-40e7483d7aaa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:40.332367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:40.333030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.333315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.333597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.334854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=9930e234fb15434b881890a315de561e, user.id=8d3febf927af4782bc2aa5631cc4dde7) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:40.334854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:40.336234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.342954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 91/182] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/role_assignments?user.id=8d3febf927af4782bc2aa5631cc4dde7&scope.project.id=9930e234fb15434b881890a315de561e&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:40.343573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.391047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15166deb-3979-43db-a123-ddeabdd1d996 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:40.436693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0615be8c-a4bc-4141-ab06-f1aaf36b2485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:40.437598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615be8c-a4bc-4141-ab06-f1aaf36b2485 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.437887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615be8c-a4bc-4141-ab06-f1aaf36b2485 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.438170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0615be8c-a4bc-4141-ab06-f1aaf36b2485 None admin] PATH_INFO: `/v3/users/b6a6d9884a694175acc37e8931c9c9ed` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.456648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615be8c-a4bc-4141-ab06-f1aaf36b2485 None admin] RBAC: Authorizing `identity:get_user(user_id=b6a6d9884a694175acc37e8931c9c9ed)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:40.458158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0615be8c-a4bc-4141-ab06-f1aaf36b2485 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.460039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 92/183] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/users/b6a6d9884a694175acc37e8931c9c9ed => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:40.461331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a12296-317f-43e3-8980-4343b928bb47 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.461625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a12296-317f-43e3-8980-4343b928bb47 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.461905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a12296-317f-43e3-8980-4343b928bb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.469677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:40.478725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f258552-92af-48ef-bd4d-32d3c0e52ba3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:40.481769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-93a12296-317f-43e3-8980-4343b928bb47 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:40.483004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-93a12296-317f-43e3-8980-4343b928bb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:40.522541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4ad75-e232-4d66-8f03-f93e410fe019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:40.523436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ad75-e232-4d66-8f03-f93e410fe019 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.523720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ad75-e232-4d66-8f03-f93e410fe019 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.524023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4ad75-e232-4d66-8f03-f93e410fe019 None admin] PATH_INFO: `/v3/users/646457ebc2974d8dad3b9b7800cf8ac7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.524591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:40.525984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4ad75-e232-4d66-8f03-f93e410fe019 None admin] RBAC: Authorizing `identity:get_user(user_id=646457ebc2974d8dad3b9b7800cf8ac7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:40.527479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4ad75-e232-4d66-8f03-f93e410fe019 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.529216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 92/184] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/users/646457ebc2974d8dad3b9b7800cf8ac7 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:40.530676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f43ed-2050-4bce-a4a1-613d3b223fc9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.530968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f43ed-2050-4bce-a4a1-613d3b223fc9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.531258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927f43ed-2050-4bce-a4a1-613d3b223fc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.556964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-927f43ed-2050-4bce-a4a1-613d3b223fc9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:40.556964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-927f43ed-2050-4bce-a4a1-613d3b223fc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:40.570769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 93/185] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:40.571654 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a12296-317f-43e3-8980-4343b928bb47 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.646199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 93/186] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:40.651785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f43ed-2050-4bce-a4a1-613d3b223fc9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.693259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a12296-317f-43e3-8980-4343b928bb47 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:40.728294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:40.728876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.729195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.729478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.756836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:40.766807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927f43ed-2050-4bce-a4a1-613d3b223fc9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:40.770369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:40.801441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:40.801655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:40.802092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:40.802376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:40.802655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:40.803122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.868068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:40.869535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:40.900369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 94/187] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:40.904644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.957133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:40.966791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 94/188] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:40.966791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:40.974634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:41.006167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37c4354-a7a9-4f1b-a266-7fe7009c8a6b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.039718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17441eb1-86dc-42fd-951b-597fea122f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.040562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17441eb1-86dc-42fd-951b-597fea122f60 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.040851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17441eb1-86dc-42fd-951b-597fea122f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.041131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17441eb1-86dc-42fd-951b-597fea122f60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.051068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17441eb1-86dc-42fd-951b-597fea122f60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-17441eb1-86dc-42fd-951b-597fea122f60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:41.059599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:41.060514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:41.061299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 95/189] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:41.062205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534d677-a568-4805-bca9-793fa416367e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.062500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534d677-a568-4805-bca9-793fa416367e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.062784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c534d677-a568-4805-bca9-793fa416367e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.064100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 96/190] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:41.065128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534d677-a568-4805-bca9-793fa416367e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.075031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8957e28-4359-47d7-8c90-3cf2b24ad460 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.110738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.112371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.112825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.113248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.115019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.117588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.125274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 95/191] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:41.125274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.173914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c534d677-a568-4805-bca9-793fa416367e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.186873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:41.203369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:41.210124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.210995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.211272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.211545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.212749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.214213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.216119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 97/192] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:41.216679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.231440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75cf4f5-aa75-4780-a8ca-48e5c9235752 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.265087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc158f9-ab21-4d5e-8f1f-db22c67f81a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.265876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc158f9-ab21-4d5e-8f1f-db22c67f81a2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.266160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc158f9-ab21-4d5e-8f1f-db22c67f81a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.266436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc158f9-ab21-4d5e-8f1f-db22c67f81a2 None admin] PATH_INFO: `/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.267609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc158f9-ab21-4d5e-8f1f-db22c67f81a2 None admin] RBAC: Authorizing `identity:get_project(project_id=1e067025a1ba4c6c8b3f38213caf81c8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.269128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc158f9-ab21-4d5e-8f1f-db22c67f81a2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.271005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 96/193] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/projects/1e067025a1ba4c6c8b3f38213caf81c8 => generated 312 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:41.271985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdaba9-edb2-4de8-9877-0eef6ca1ac4e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.272614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdaba9-edb2-4de8-9877-0eef6ca1ac4e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.272614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fdaba9-edb2-4de8-9877-0eef6ca1ac4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.288859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-83fdaba9-edb2-4de8-9877-0eef6ca1ac4e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:41.290031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-83fdaba9-edb2-4de8-9877-0eef6ca1ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:41.320034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ae72da-57a0-4689-9e52-8a2d9c15ceac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.352768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.353436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.353722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.354000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.354310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 97/194] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:41.354761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:41.355073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.355185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdaba9-edb2-4de8-9877-0eef6ca1ac4e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.356570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.374894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 98/195] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:41.375978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.419387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:41.433383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:41.461153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fdaba9-edb2-4de8-9877-0eef6ca1ac4e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.474231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cbf050-8cd6-47e7-ad16-2fe69ac3d514 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.496939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.497614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.497894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.498173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.498880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.500403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.510334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.511053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.511343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.511627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.512506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=b6a6d9884a694175acc37e8931c9c9ed) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:41.513410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.514461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.520987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 99/196] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/role_assignments?user.id=b6a6d9884a694175acc37e8931c9c9ed&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:41.521772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.522415 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 98/197] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:12:41 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:41.523220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.664734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04da907f-3ebe-47c4-be65-564c4335439c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.669786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e300a4-c337-44ef-b72f-e67fa56e4d2e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.687058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:41.698990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:41.716422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.717103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.717391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.717674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.718515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=1e067025a1ba4c6c8b3f38213caf81c8, user.id=646457ebc2974d8dad3b9b7800cf8ac7) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:41.718832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.720383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.725518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.726797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.727305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.727798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.727985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 100/198] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/role_assignments?user.id=646457ebc2974d8dad3b9b7800cf8ac7&scope.project.id=1e067025a1ba4c6c8b3f38213caf81c8&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:41.728798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.766710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:41.776493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:41.777419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:41.778229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:41.781345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 99/199] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:41.788205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.869431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce936268-fa9f-4520-8fc1-944618315f48 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:41.923523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:41.924556 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:41.924842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:41.925156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] PATH_INFO: `/v3/users/b6a6d9884a694175acc37e8931c9c9ed` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:41.926511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] RBAC: Authorizing `identity:get_user(user_id=b6a6d9884a694175acc37e8931c9c9ed)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:41.928009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:41.930656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 101/200] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/users/b6a6d9884a694175acc37e8931c9c9ed => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:41.936973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:41.957543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd611e17-0bdc-4821-a5c5-3a3cab4cd08b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:42.004041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:42.005186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:42.005703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:42.006215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:42.007283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:42.008459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:42.010610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:42.012820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:42.024216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 100/201] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:42.029419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:42.072240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e98c4f-c009-4240-974f-b246789e3221 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:42.075498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:42.117182 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:42.118057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:42.118446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:42.119159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:42.165233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1c1a06-b104-450b-9c22-6b5c51facace None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:42.193291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:42.195008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:42.212359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:42.213255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:42.213538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:42.213814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] PATH_INFO: `/v3/users/df7c8088fc764cf4899956ca28fc4463` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:42.217429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] RBAC: Authorizing `identity:get_user(user_id=df7c8088fc764cf4899956ca28fc4463)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:42.218901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:42.220780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 101/202] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:42 2022] GET /identity/v3/users/df7c8088fc764cf4899956ca28fc4463 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:42.229259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:42.282215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:42.370823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 102/203] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:42.377121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:42.417022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead63964-9c0b-4cee-ab61-3b7d9b5ddc37 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:42.420119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:42.479571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:42.480261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:42.480548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:42.480831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:42.568072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:42.570190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:42.595578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f012b677-f1b1-4f6f-9f4f-cab9a9a6b433 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:42.681800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:42.682682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:42.682963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:42.683238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:42.685427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:42.686896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:42.688635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 103/204] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:42 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:42.701248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:42.715929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 102/205] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:42.721450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:42.732035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:42.748248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:42.849421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c1a377-d13e-4329-a69c-133dbde313d1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:42.886677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:42.887349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:42.887640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:42.887953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:42.888798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] RBAC: Adding query filter params (role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=b6a6d9884a694175acc37e8931c9c9ed) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:42.889119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:42.890568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:42.897123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 104/206] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:42 2022] GET /identity/v3/role_assignments?user.id=b6a6d9884a694175acc37e8931c9c9ed&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=7673131020174664baa4d10a6f38ce2e => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:42.980020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40083387-360e-496b-a9ed-b3ab45ea9d98 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:43.034251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da73a62-3d61-4891-911e-1e9c582285df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:43.035193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73a62-3d61-4891-911e-1e9c582285df None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:43.035528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73a62-3d61-4891-911e-1e9c582285df None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:43.035835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da73a62-3d61-4891-911e-1e9c582285df None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:43.037069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da73a62-3d61-4891-911e-1e9c582285df None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:43.038567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da73a62-3d61-4891-911e-1e9c582285df None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:43.040748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 103/207] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:42 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:43.046449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:43.150244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:43.163029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3282ae05-38a4-414b-8c50-a88bfdf65eca None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:43.197524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc3958-ae87-4b4e-a644-dfacaba523c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:43.198190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc3958-ae87-4b4e-a644-dfacaba523c4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:43.198475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc3958-ae87-4b4e-a644-dfacaba523c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:43.198749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc3958-ae87-4b4e-a644-dfacaba523c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:43.199584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc3958-ae87-4b4e-a644-dfacaba523c4 None admin] RBAC: Adding query filter params (role.id=70a666d4a1c5476790ac8f65c8a167f3, scope.project.id=9930e234fb15434b881890a315de561e, user.id=df7c8088fc764cf4899956ca28fc4463) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:43.200850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc3958-ae87-4b4e-a644-dfacaba523c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:43.201378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7cc3958-ae87-4b4e-a644-dfacaba523c4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:43.216087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 105/208] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:43 2022] GET /identity/v3/role_assignments?user.id=df7c8088fc764cf4899956ca28fc4463&scope.project.id=9930e234fb15434b881890a315de561e&role.id=70a666d4a1c5476790ac8f65c8a167f3 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:43.987067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1cabb-21e6-460a-ae6e-ae4efe1ff9dd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:43.988118 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1cabb-21e6-460a-ae6e-ae4efe1ff9dd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:43.988223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c1cabb-21e6-460a-ae6e-ae4efe1ff9dd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:43.989558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 104/209] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:43.994807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9b378-2592-416e-b5fa-57e9e93cc147 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:43.995326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9b378-2592-416e-b5fa-57e9e93cc147 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:43.995837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e9b378-2592-416e-b5fa-57e9e93cc147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.014409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b0e9b378-2592-416e-b5fa-57e9e93cc147 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:44.016964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b0e9b378-2592-416e-b5fa-57e9e93cc147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:44.018382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6debf33-e75b-442c-9720-4f82f70f63ac None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.019203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6debf33-e75b-442c-9720-4f82f70f63ac None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.020028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6debf33-e75b-442c-9720-4f82f70f63ac None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.022393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 105/210] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:44.027993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b01d7-a1b0-4283-8c16-d80827f57044 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.028531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b01d7-a1b0-4283-8c16-d80827f57044 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.029018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0b01d7-a1b0-4283-8c16-d80827f57044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.046902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2a0b01d7-a1b0-4283-8c16-d80827f57044 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:44.048111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2a0b01d7-a1b0-4283-8c16-d80827f57044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:44.099718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 106/211] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:44.122966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9b378-2592-416e-b5fa-57e9e93cc147 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.135664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 106/212] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:44.157154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b01d7-a1b0-4283-8c16-d80827f57044 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.160120 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:44.214170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:44.233770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e9b378-2592-416e-b5fa-57e9e93cc147 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.269455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e9321f-af25-45dd-8e02-57ebd008c5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.270658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9321f-af25-45dd-8e02-57ebd008c5a6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.271142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9321f-af25-45dd-8e02-57ebd008c5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.271630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e9321f-af25-45dd-8e02-57ebd008c5a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.275302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0b01d7-a1b0-4283-8c16-d80827f57044 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.282406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e9321f-af25-45dd-8e02-57ebd008c5a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-47e9321f-af25-45dd-8e02-57ebd008c5a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:44.291133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:44.292042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:44.292815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:44.293982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 107/213] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:12:44.295447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b3046-66af-428f-ae52-7b4e07555254 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.295964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b3046-66af-428f-ae52-7b4e07555254 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.296489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9b3046-66af-428f-ae52-7b4e07555254 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.297976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 108/214] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:44.298837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b3046-66af-428f-ae52-7b4e07555254 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.312280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2960ec32-5d8e-496d-89fa-9026ad016e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.313383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960ec32-5d8e-496d-89fa-9026ad016e66 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.313804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960ec32-5d8e-496d-89fa-9026ad016e66 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.314478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2960ec32-5d8e-496d-89fa-9026ad016e66 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.326126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2960ec32-5d8e-496d-89fa-9026ad016e66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2960ec32-5d8e-496d-89fa-9026ad016e66 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:44.335331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:44.336367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:44.337202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:44.339381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 107/215] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:12:44.340702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e3a27-8c14-485c-9996-f083acf6fda4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.341126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e3a27-8c14-485c-9996-f083acf6fda4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.341651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e3a27-8c14-485c-9996-f083acf6fda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.358696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f73e3a27-8c14-485c-9996-f083acf6fda4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:44.360093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f73e3a27-8c14-485c-9996-f083acf6fda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:44.408619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9b3046-66af-428f-ae52-7b4e07555254 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.422349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:44.433379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 108/216] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:44.436564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e3a27-8c14-485c-9996-f083acf6fda4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.449517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.450945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.451712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.452239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.453519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:44.454468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:44.457225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:44.470979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 109/217] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:44.472103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.475873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:44.545717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e3a27-8c14-485c-9996-f083acf6fda4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.582852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.583568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.583896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.584174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.584917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:44.585222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:44.585541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b99cae-5a1e-4ce5-a505-de0b7d898b86 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.586612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:44.594790 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 109/218] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:44.595407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.626578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.627361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.627645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.628502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.629684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:44.631131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:44.632944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 110/219] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:44.633550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.655349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:44.702922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c77d604-ddff-47dd-bfb7-8272d6763359 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.710579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:44.736757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.738453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.738738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.739008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] PATH_INFO: `/v3/users/08d09c5c92af4c5aab225c91131cd6e0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.744115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911904f-8bab-4ad5-8b30-2a1253d01f50 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.753875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] RBAC: Authorizing `identity:get_user(user_id=08d09c5c92af4c5aab225c91131cd6e0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:44.755352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:44.757092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 110/220] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/users/08d09c5c92af4c5aab225c91131cd6e0 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:44.762145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.779531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.780785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.781245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.781719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] PATH_INFO: `/v3/users/b6a6d9884a694175acc37e8931c9c9ed` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.783189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] RBAC: Authorizing `identity:get_user(user_id=b6a6d9884a694175acc37e8931c9c9ed)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:44.784889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:44.786905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 111/221] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/users/b6a6d9884a694175acc37e8931c9c9ed => generated 283 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:44.787867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.864776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887a03e1-8d68-4c49-88b1-54fb48e3d2de None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.877687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:44.892803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1223ee4b-f060-43bc-8667-828302112045 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:44.903318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.904329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.904614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.905040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.905751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:44.907203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:44.927413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 111/222] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:12:44 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:44.928035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ee179-a439-4308-a855-556a37b22123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:44.928682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:44.929049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee179-a439-4308-a855-556a37b22123 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:44.929493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee179-a439-4308-a855-556a37b22123 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:44.929944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ee179-a439-4308-a855-556a37b22123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:44.948747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:45.016377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ee179-a439-4308-a855-556a37b22123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:45.017855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ee179-a439-4308-a855-556a37b22123 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:45.074307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0d957-96fd-4ae3-9fb4-4861c3c91de6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:45.121981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53967a7-c07f-4415-bcf8-cf2c4316451a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:45.122781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53967a7-c07f-4415-bcf8-cf2c4316451a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.123066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53967a7-c07f-4415-bcf8-cf2c4316451a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.123341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53967a7-c07f-4415-bcf8-cf2c4316451a None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.124568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53967a7-c07f-4415-bcf8-cf2c4316451a None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:45.126069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53967a7-c07f-4415-bcf8-cf2c4316451a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:45.131813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 112/223] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:45.133357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edf338-6f48-4eca-8ede-8199fcd62cd9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.133657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edf338-6f48-4eca-8ede-8199fcd62cd9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.133940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60edf338-6f48-4eca-8ede-8199fcd62cd9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.135424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 113/224] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:45.136081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edf338-6f48-4eca-8ede-8199fcd62cd9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:45.152774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 112/225] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:45.153912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3827f6d-148c-41fa-a52a-912368cff4a6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.154253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3827f6d-148c-41fa-a52a-912368cff4a6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.154594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3827f6d-148c-41fa-a52a-912368cff4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.172663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a3827f6d-148c-41fa-a52a-912368cff4a6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:45.173860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3827f6d-148c-41fa-a52a-912368cff4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:45.180440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:45.214018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:45.257932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 113/226] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:45.259821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3827f6d-148c-41fa-a52a-912368cff4a6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:45.264937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60edf338-6f48-4eca-8ede-8199fcd62cd9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:45.304774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:45.305678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.305960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.306235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/b6a6d9884a694175acc37e8931c9c9ed/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.308239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] RBAC: Authorizing `identity:create_grant(project_id=0b3bdd01c0a249b5a7dfce37407324f1, user_id=b6a6d9884a694175acc37e8931c9c9ed, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:45.310217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:45.327625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 114/227] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:45 2022] PUT /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/b6a6d9884a694175acc37e8931c9c9ed/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:45.328545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:45.446319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3827f6d-148c-41fa-a52a-912368cff4a6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:45.448243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:45.456056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2381f0e7-9a0e-491a-b661-8868f6e79bed None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:45.495117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:45.495983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.496268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.496543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.509320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:45.518122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:45.519092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:45.519894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 115/228] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:12:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:45.522863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:45.527795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.527795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.527795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.529009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:45.530381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:45.532366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 114/229] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:45 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:45.532866 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:45.539680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:45.586164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:45.646576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525b0ed-377c-46c7-a2ad-8046372bb2ba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:45.682925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:45.683961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.684451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.684924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.685995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=08d09c5c92af4c5aab225c91131cd6e0) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:45.686509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:45.688238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:45.693281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15765edf-f615-4923-b1f0-d9bc6188f630 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:45.699941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 115/230] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:45 2022] GET /identity/v3/role_assignments?user.id=08d09c5c92af4c5aab225c91131cd6e0&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:45.758970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:45.759663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.759975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.760253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.761016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:45.761324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:45.762744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:45.780933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 116/231] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:12:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:45.786778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:45.926387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5932ffd-85ac-48bc-8e4f-2be99f39c0c2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:45.927967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:45.970390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:45.971202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:45.972024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:45.972316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] PATH_INFO: `/v3/users/b6a6d9884a694175acc37e8931c9c9ed` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:45.973542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] RBAC: Authorizing `identity:get_user(user_id=b6a6d9884a694175acc37e8931c9c9ed)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:45.975012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:45.977007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 116/232] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:45 2022] GET /identity/v3/users/b6a6d9884a694175acc37e8931c9c9ed => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:45.982111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:46.003372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:46.091368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e52e71d-ba4f-4149-87c9-1c11ddcf9f84 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:46.126240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d90a30-07dc-4a4e-b0db-dc9f849baf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:46.127130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d90a30-07dc-4a4e-b0db-dc9f849baf22 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:46.127374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d90a30-07dc-4a4e-b0db-dc9f849baf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:46.127643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d90a30-07dc-4a4e-b0db-dc9f849baf22 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:46.128891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d90a30-07dc-4a4e-b0db-dc9f849baf22 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:46.130347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d90a30-07dc-4a4e-b0db-dc9f849baf22 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:46.132372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 117/233] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:45 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:46.140792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:46.263656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a669b502-6ae3-4499-a36a-a652ea5e74ea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:46.309444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075341d7-0840-4bf8-824b-0613ae8a9296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:46.310445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075341d7-0840-4bf8-824b-0613ae8a9296 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:46.310732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075341d7-0840-4bf8-824b-0613ae8a9296 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:46.311008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075341d7-0840-4bf8-824b-0613ae8a9296 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/b6a6d9884a694175acc37e8931c9c9ed/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:46.317549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075341d7-0840-4bf8-824b-0613ae8a9296 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b3bdd01c0a249b5a7dfce37407324f1, user_id=b6a6d9884a694175acc37e8931c9c9ed, role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:46.319012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075341d7-0840-4bf8-824b-0613ae8a9296 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:46.342678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 117/234] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:46 2022] PUT /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/b6a6d9884a694175acc37e8931c9c9ed/roles/7673131020174664baa4d10a6f38ce2e => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:47.520352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc0526-7c00-442f-95c5-197af3f5601d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:47.521207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc0526-7c00-442f-95c5-197af3f5601d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:47.521675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc0526-7c00-442f-95c5-197af3f5601d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:47.523508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 118/235] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:47.528542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3c81-9095-4c99-89cf-64704bb17ad7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:47.529325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3c81-9095-4c99-89cf-64704bb17ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:47.529817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ae3c81-9095-4c99-89cf-64704bb17ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:47.556720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a0ae3c81-9095-4c99-89cf-64704bb17ad7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:47.558225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0ae3c81-9095-4c99-89cf-64704bb17ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:47.594379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:47.640542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 118/236] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:47.681220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc0526-7c00-442f-95c5-197af3f5601d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:47.701164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936c48-f3b2-4cdf-a5f4-b27de96079e7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:47.701164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936c48-f3b2-4cdf-a5f4-b27de96079e7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:47.701164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2936c48-f3b2-4cdf-a5f4-b27de96079e7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:47.701758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 119/237] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:47.707322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8a9e1-2a2c-4332-8066-bbab6caf301c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:47.707834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8a9e1-2a2c-4332-8066-bbab6caf301c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:47.708280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a8a9e1-2a2c-4332-8066-bbab6caf301c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:47.725780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e9a8a9e1-2a2c-4332-8066-bbab6caf301c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:47.727222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9a8a9e1-2a2c-4332-8066-bbab6caf301c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:47.803942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 120/238] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:47.820315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8a9e1-2a2c-4332-8066-bbab6caf301c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:47.852348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:47.880241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc0526-7c00-442f-95c5-197af3f5601d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:47.926144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:47.942305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:47.943133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:47.943417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:47.943696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:47.945229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:47.946756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:47.948731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 119/239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:47.954980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a8a9e1-2a2c-4332-8066-bbab6caf301c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:47.959240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.009775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c8269-1987-4776-b6f6-69f5cd45ed39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.010596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c8269-1987-4776-b6f6-69f5cd45ed39 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.010885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c8269-1987-4776-b6f6-69f5cd45ed39 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.011166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c8269-1987-4776-b6f6-69f5cd45ed39 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.025614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c8269-1987-4776-b6f6-69f5cd45ed39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2c7c8269-1987-4776-b6f6-69f5cd45ed39 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:48.036159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:48.037028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:48.037788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 121/240] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:12:48.038658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af23c1-3f12-44ad-94d6-344614d7c29c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.039095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af23c1-3f12-44ad-94d6-344614d7c29c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.044162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af23c1-3f12-44ad-94d6-344614d7c29c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.048567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 122/241] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:48.051318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af23c1-3f12-44ad-94d6-344614d7c29c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.107039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae90093-7b5b-4923-9cfb-9a788fc2a4ff None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:48.149236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0189e77-32c1-452a-850f-450681ae09d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.150896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0189e77-32c1-452a-850f-450681ae09d8 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.151344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0189e77-32c1-452a-850f-450681ae09d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.151781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0189e77-32c1-452a-850f-450681ae09d8 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.152657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0189e77-32c1-452a-850f-450681ae09d8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:48.154301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0189e77-32c1-452a-850f-450681ae09d8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:48.180251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 120/242] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:12:47 2022] POST /identity/v3/users => generated 293 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:48.181759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45adf-63bf-4f4b-b20a-c41b35be09fa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.182179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45adf-63bf-4f4b-b20a-c41b35be09fa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.182618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b45adf-63bf-4f4b-b20a-c41b35be09fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.201492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af23c1-3f12-44ad-94d6-344614d7c29c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:48.215263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-11b45adf-63bf-4f4b-b20a-c41b35be09fa None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:48.216716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-11b45adf-63bf-4f4b-b20a-c41b35be09fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:48.221102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:48.243969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0b1bcc-1a2f-4210-a900-8fef0db9316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.244671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b1bcc-1a2f-4210-a900-8fef0db9316f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.244963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b1bcc-1a2f-4210-a900-8fef0db9316f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.245243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0b1bcc-1a2f-4210-a900-8fef0db9316f None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.246022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b1bcc-1a2f-4210-a900-8fef0db9316f None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:48.246336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b1bcc-1a2f-4210-a900-8fef0db9316f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:48.247798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0b1bcc-1a2f-4210-a900-8fef0db9316f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:48.258239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 123/243] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:48.259890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192e17f-f345-409f-b77c-326fdc177d36 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.260183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192e17f-f345-409f-b77c-326fdc177d36 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.260538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0192e17f-f345-409f-b77c-326fdc177d36 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.265979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 124/244] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:48.266844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192e17f-f345-409f-b77c-326fdc177d36 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.291759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:48.316437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 121/245] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:48.320126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427dbf-4761-476c-aede-a54c868a3c8b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.320410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427dbf-4761-476c-aede-a54c868a3c8b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.322172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37427dbf-4761-476c-aede-a54c868a3c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.347546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-37427dbf-4761-476c-aede-a54c868a3c8b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:48.349973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-37427dbf-4761-476c-aede-a54c868a3c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:48.400677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0192e17f-f345-409f-b77c-326fdc177d36 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:48.430511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 122/246] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:48.431484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427dbf-4761-476c-aede-a54c868a3c8b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.449242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.450727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.451668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.452168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] PATH_INFO: `/v3/users/b5d0f422d7c944f5a52ae581e2eb18af` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.469687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] RBAC: Authorizing `identity:get_user(user_id=b5d0f422d7c944f5a52ae581e2eb18af)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:48.471338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:48.473290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 125/247] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:48 2022] GET /identity/v3/users/b5d0f422d7c944f5a52ae581e2eb18af => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:48.473906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.520376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:48.563132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37427dbf-4761-476c-aede-a54c868a3c8b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:48.569950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:48.609278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6daeea8-e066-4785-a5de-db22a8aaa7bb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:48.618300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.619125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.619411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.619954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.633588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:48.641624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:48.642566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:48.643901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 123/248] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:12:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:48.644560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.646601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.647411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.647677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.652148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.666940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:48.677488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:48.678408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:48.679189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:48.679840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 126/249] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:48.680507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.754226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74c53c5-6f4e-4621-978b-b8be79020443 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:48.784472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:48.816657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cae4f5-0943-44c4-9e31-95cc0de25278 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:48.828467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae20b0-ba4d-40c0-b9ca-d52371003da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.829151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae20b0-ba4d-40c0-b9ca-d52371003da3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.829447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae20b0-ba4d-40c0-b9ca-d52371003da3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.829731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae20b0-ba4d-40c0-b9ca-d52371003da3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.850734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:48.855200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:48.856023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:48.856410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:48.856786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:48.857655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:48.858066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:48.859564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:48.871481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 127/250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:48.872714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:48.985820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae20b0-ba4d-40c0-b9ca-d52371003da3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:48.987586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ae20b0-ba4d-40c0-b9ca-d52371003da3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.013356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38944ecc-aeef-41f2-a477-e6f0539dddd4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.060297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.061229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.061909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.062197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.062966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:49.063280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.066058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.078066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 128/251] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:49.079170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.123376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 124/252] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:49.124799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766356-0037-4baf-ac54-f93de8014027 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.125087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766356-0037-4baf-ac54-f93de8014027 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.125359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42766356-0037-4baf-ac54-f93de8014027 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.126788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 125/253] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:49.132123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766356-0037-4baf-ac54-f93de8014027 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.156391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:49.168301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:49.210107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a83da8-3cd5-426b-b9a6-b082a2b26201 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.245465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.246260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.246773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.246903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] PATH_INFO: `/v3/users/b6a6d9884a694175acc37e8931c9c9ed` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.248077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] RBAC: Authorizing `identity:get_user(user_id=b6a6d9884a694175acc37e8931c9c9ed)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.249745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.250885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42766356-0037-4baf-ac54-f93de8014027 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.251403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 129/254] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/users/b6a6d9884a694175acc37e8931c9c9ed => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:49.252287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.290330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a05b9f3-a8d3-4ee5-8796-83e0896d1569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.291608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b9f3-a8d3-4ee5-8796-83e0896d1569 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.292127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b9f3-a8d3-4ee5-8796-83e0896d1569 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.292612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a05b9f3-a8d3-4ee5-8796-83e0896d1569 None admin] PATH_INFO: `/v3/users/08d09c5c92af4c5aab225c91131cd6e0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.294128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05b9f3-a8d3-4ee5-8796-83e0896d1569 None admin] RBAC: Authorizing `identity:get_user(user_id=08d09c5c92af4c5aab225c91131cd6e0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.295927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a05b9f3-a8d3-4ee5-8796-83e0896d1569 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.302863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 126/255] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/users/08d09c5c92af4c5aab225c91131cd6e0 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:49.304544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9fbd4-f1e2-4505-a10e-aa57be0ce7a8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.305026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9fbd4-f1e2-4505-a10e-aa57be0ce7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.305513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9fbd4-f1e2-4505-a10e-aa57be0ce7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.328516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-65c9fbd4-f1e2-4505-a10e-aa57be0ce7a8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:49.330039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-65c9fbd4-f1e2-4505-a10e-aa57be0ce7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:49.358431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d941329-aa71-4f53-a1c1-26cec1f6efc9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.383602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:49.396280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.397253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.397544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.397823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.399117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.400605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.402541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 130/256] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:49.408097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.428232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 127/257] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:49.429342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9fbd4-f1e2-4505-a10e-aa57be0ce7a8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.476430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:49.545579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abeb6f4-ffbb-4abb-b55c-c8d8b6c0eb7a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.570119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9fbd4-f1e2-4505-a10e-aa57be0ce7a8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.582623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.583913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.584393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.584862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.609713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.610683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.610981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.611259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.613138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.614600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.621328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 128/258] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:49.621328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.660016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:49.680963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.683620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.724153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:49.731280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aba0d3-7004-4d3c-9a09-546375c45541 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.769775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.770343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.770740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.771109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.772178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] RBAC: Adding query filter params (role.id=70a666d4a1c5476790ac8f65c8a167f3, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=b5d0f422d7c944f5a52ae581e2eb18af) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:49.772592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.774130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.780335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 129/259] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/role_assignments?user.id=b5d0f422d7c944f5a52ae581e2eb18af&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=70a666d4a1c5476790ac8f65c8a167f3 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:49.781162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.797153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 131/260] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:49.798415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.894430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7a2cd6-fee3-474a-9f57-f13a53dfc602 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.900381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:49.911459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef428b39-e6a8-4e4e-81e6-6b0b01cf16d0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:49.934775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.935682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.936087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.936458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.949078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:49.950087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:49.951173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:49.951592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:49.952047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/08d09c5c92af4c5aab225c91131cd6e0/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:49.954230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b3bdd01c0a249b5a7dfce37407324f1, user_id=08d09c5c92af4c5aab225c91131cd6e0, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:49.956377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:49.960375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:49.961243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:49.962018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 130/261] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:49.962687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.979230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 132/262] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:49 2022] PUT /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/08d09c5c92af4c5aab225c91131cd6e0/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:49.980679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:49.988194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:50.086028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9303e70b-f5e6-462e-8a68-1c417eb36c54 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:50.101299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5280f8b5-1aa8-48f9-8640-bf2a21d67cc9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:50.126404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:50.127394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:50.127722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:50.128019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:50.129304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:50.130840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:50.132907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 131/263] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:50.140792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:50.145884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:50.146989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:50.147517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:50.148026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:50.148921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:50.149371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:50.151038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:50.161905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 133/264] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:12:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:50.181048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:50.220719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:50.348579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912a9d1c-da97-4785-a10a-d86841bd427d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:50.364272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:50.390310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:50.391200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:50.391482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:50.391773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] PATH_INFO: `/v3/users/b6a6d9884a694175acc37e8931c9c9ed` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:50.395189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] RBAC: Authorizing `identity:get_user(user_id=b6a6d9884a694175acc37e8931c9c9ed)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:50.396676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:50.398327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 134/265] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:50 2022] GET /identity/v3/users/b6a6d9884a694175acc37e8931c9c9ed => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:50.404245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce2803c-3af7-4c5d-a3a5-b0cf45bd2c1e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:50.406529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:50.473244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:50.473911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:50.474193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:50.474468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:50.475301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=b6a6d9884a694175acc37e8931c9c9ed) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:50.475616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:50.477104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:50.483946 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 132/266] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:50 2022] GET /identity/v3/role_assignments?user.id=b6a6d9884a694175acc37e8931c9c9ed&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:50.552521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:50.569285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8797df6-461e-489d-a157-47eae46ce3ec None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:50.611446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330b34b0-6925-4628-9966-e8bf78fe019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:50.612623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b34b0-6925-4628-9966-e8bf78fe019a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:50.612878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b34b0-6925-4628-9966-e8bf78fe019a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:50.613156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330b34b0-6925-4628-9966-e8bf78fe019a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:50.699103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b34b0-6925-4628-9966-e8bf78fe019a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:50.700585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330b34b0-6925-4628-9966-e8bf78fe019a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:50.785651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:50.812556 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 135/267] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:50.818390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:50.854092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9754a9-13b2-4fd9-864c-daaa1026a622 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:50.854417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9754a9-13b2-4fd9-864c-daaa1026a622 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:50.854658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9754a9-13b2-4fd9-864c-daaa1026a622 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:50.856462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 136/268] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:50.867237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c307e3c-2595-4417-bec2-36f5d64e38b2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:50.867477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c307e3c-2595-4417-bec2-36f5d64e38b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:50.867765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c307e3c-2595-4417-bec2-36f5d64e38b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:50.901473 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0c307e3c-2595-4417-bec2-36f5d64e38b2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:50.902712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c307e3c-2595-4417-bec2-36f5d64e38b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:50.974188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befee0b9-1e9a-488b-8b6c-c16876f3b389 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.016188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 137/269] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:51.020490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.021599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.022084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.022559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.023971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:51.025610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:51.027734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 133/270] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:50 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:51.034309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c307e3c-2595-4417-bec2-36f5d64e38b2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:51.051572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:51.078759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:51.181445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f59118-957b-43ce-99f9-7e02955a2fbc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.193277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c307e3c-2595-4417-bec2-36f5d64e38b2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.207027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:51.221773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9397dd65-f451-428a-a26c-538bf59b8626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.222581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397dd65-f451-428a-a26c-538bf59b8626 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.222861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397dd65-f451-428a-a26c-538bf59b8626 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.223134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9397dd65-f451-428a-a26c-538bf59b8626 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.237387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9397dd65-f451-428a-a26c-538bf59b8626 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:51.239618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9cb139-71d6-48cf-9146-bab17ed28f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.240540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cb139-71d6-48cf-9146-bab17ed28f71 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.241033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cb139-71d6-48cf-9146-bab17ed28f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.241528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9cb139-71d6-48cf-9146-bab17ed28f71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.242574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cb139-71d6-48cf-9146-bab17ed28f71 None admin] RBAC: Adding query filter params (role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=b6a6d9884a694175acc37e8931c9c9ed) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:51.243106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cb139-71d6-48cf-9146-bab17ed28f71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:51.244774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9cb139-71d6-48cf-9146-bab17ed28f71 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9397dd65-f451-428a-a26c-538bf59b8626 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:51.245313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:51.246175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:51.247365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 134/271] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:51.248292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f9f4a-3508-4c1b-b3a9-96e56365639d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.248528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f9f4a-3508-4c1b-b3a9-96e56365639d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.248716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0f9f4a-3508-4c1b-b3a9-96e56365639d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.250194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 135/272] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:51.255000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f9f4a-3508-4c1b-b3a9-96e56365639d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:51.259841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 138/273] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:51 2022] GET /identity/v3/role_assignments?user.id=b6a6d9884a694175acc37e8931c9c9ed&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=7673131020174664baa4d10a6f38ce2e => generated 608 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:51.262265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8e5cf-0742-458e-bfdb-a243c09a2a8e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.262773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8e5cf-0742-458e-bfdb-a243c09a2a8e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.263256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d8e5cf-0742-458e-bfdb-a243c09a2a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.294993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-63d8e5cf-0742-458e-bfdb-a243c09a2a8e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:51.296244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-63d8e5cf-0742-458e-bfdb-a243c09a2a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:51.416117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:51.443915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 139/274] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:51.455116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0f9f4a-3508-4c1b-b3a9-96e56365639d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.468845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8e5cf-0742-458e-bfdb-a243c09a2a8e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:51.493236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.494251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.494769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.495393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.496469 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:51.496983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:51.498777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:51.510225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 136/275] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:51.517658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:51.576278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d8e5cf-0742-458e-bfdb-a243c09a2a8e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.579477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:51.620502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.621342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.621711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.621990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.623310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:51.625558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:51.627346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 140/276] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:51.633649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:51.652170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984fd722-c77c-4365-8c76-1e01ba08ba5e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.695513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.697128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.697554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.698247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] PATH_INFO: `/v3/users/f0ac3957137e4a35bb1b362c881ff4aa` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.714153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:51.717097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] RBAC: Authorizing `identity:get_user(user_id=f0ac3957137e4a35bb1b362c881ff4aa)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:51.719065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:51.722146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 137/277] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:51 2022] GET /identity/v3/users/f0ac3957137e4a35bb1b362c881ff4aa => generated 293 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:51.727580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:51.774081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa9117-7479-4b4d-a5fd-552961a41003 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.818368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de70fc28-d05c-40a6-9da2-bb3fff030e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.819416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70fc28-d05c-40a6-9da2-bb3fff030e68 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.819912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70fc28-d05c-40a6-9da2-bb3fff030e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.820360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de70fc28-d05c-40a6-9da2-bb3fff030e68 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.821249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70fc28-d05c-40a6-9da2-bb3fff030e68 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:51.823088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de70fc28-d05c-40a6-9da2-bb3fff030e68 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:51.845526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 141/278] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:12:51 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:51.856288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5ed7f-9b55-4a8f-8604-3a7c41376719 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:51.896996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b6e8d-0ffd-478e-86c8-5ac7cc4cc643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:51.897668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b6e8d-0ffd-478e-86c8-5ac7cc4cc643 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:51.897952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b6e8d-0ffd-478e-86c8-5ac7cc4cc643 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:51.898284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b6e8d-0ffd-478e-86c8-5ac7cc4cc643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:51.899184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b6e8d-0ffd-478e-86c8-5ac7cc4cc643 None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, user.id=f0ac3957137e4a35bb1b362c881ff4aa, scope.system=all) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:51.899498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b6e8d-0ffd-478e-86c8-5ac7cc4cc643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:51.901156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b6e8d-0ffd-478e-86c8-5ac7cc4cc643 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:51.907803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 138/279] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jul 7 10:12:51 2022] GET /identity/v3/role_assignments?user.id=f0ac3957137e4a35bb1b362c881ff4aa&scope.system=all&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:52.423306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546a1db-e74f-45f6-a4aa-21a1b4be57e9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:52.423634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546a1db-e74f-45f6-a4aa-21a1b4be57e9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:52.423900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546a1db-e74f-45f6-a4aa-21a1b4be57e9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:52.426227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 142/280] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:52.431324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e954-ca7c-41e5-b048-684812c973ad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:52.431864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e954-ca7c-41e5-b048-684812c973ad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:52.432332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67e954-ca7c-41e5-b048-684812c973ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:52.457034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3e67e954-ca7c-41e5-b048-684812c973ad None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:52.458562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3e67e954-ca7c-41e5-b048-684812c973ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:52.547098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:52.561398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 139/281] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:52.596948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546a1db-e74f-45f6-a4aa-21a1b4be57e9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:52.634675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:52.711020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29535d6-1991-45cd-a5e8-ae9f0f30c9ed None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:52.711691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29535d6-1991-45cd-a5e8-ae9f0f30c9ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:52.711998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29535d6-1991-45cd-a5e8-ae9f0f30c9ed None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:52.713841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 140/282] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:52.721630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4c905-6b31-4004-a88b-fbf9327e50f3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:52.721915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4c905-6b31-4004-a88b-fbf9327e50f3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:52.722194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b4c905-6b31-4004-a88b-fbf9327e50f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:52.733405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546a1db-e74f-45f6-a4aa-21a1b4be57e9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:52.759663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-22b4c905-6b31-4004-a88b-fbf9327e50f3 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:52.761231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-22b4c905-6b31-4004-a88b-fbf9327e50f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:52.806443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:52.807600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:52.808140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:52.808618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:52.823071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:52.833800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:52.834700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:52.835479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:52.839513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 143/283] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:52.847764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:52.917212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 141/284] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:52.964649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4c905-6b31-4004-a88b-fbf9327e50f3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:53.018015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:53.042007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e18df0c-6847-4733-a689-b2aa2e9e06e2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.113532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.114213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.114492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.114766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.115513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:53.115841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:53.117230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:53.145646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b4c905-6b31-4004-a88b-fbf9327e50f3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.147119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 144/285] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:53.149024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:53.192039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:53.217041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.218230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.219855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.220381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.242722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:53.256615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:53.257538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:53.258843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 142/286] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:53.264133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:53.378255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f218f9-0465-4f7a-a6de-f95b4d10cf6b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.379383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad1307c-42e2-4855-84e2-ac632e9e578b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.417857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.418525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.418806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.419078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.419841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:53.420147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:53.421525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:53.440298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 143/287] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:53.447071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:53.460839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.461718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.462001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.462276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] PATH_INFO: `/v3/users/b5d0f422d7c944f5a52ae581e2eb18af` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.463498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] RBAC: Authorizing `identity:get_user(user_id=b5d0f422d7c944f5a52ae581e2eb18af)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:53.464974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:53.466608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 145/288] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:53 2022] GET /identity/v3/users/b5d0f422d7c944f5a52ae581e2eb18af => generated 297 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:53.489256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:53.526201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:53.528833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:53.603162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e116a3-bb3e-4a04-b82e-f7544c796931 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.637432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.638304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.638589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.638859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] PATH_INFO: `/v3/users/08d09c5c92af4c5aab225c91131cd6e0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.640074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] RBAC: Authorizing `identity:get_user(user_id=08d09c5c92af4c5aab225c91131cd6e0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:53.641522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:53.643163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 144/289] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:53 2022] GET /identity/v3/users/08d09c5c92af4c5aab225c91131cd6e0 => generated 297 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:53.648482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:53.654735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e77b5f-129b-4e81-973f-ac85eab3fd48 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.699554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.700811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.701332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.701842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.703373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:53.705138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:53.713419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 146/290] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:53 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:53.720771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:53.756121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168acbb9-dc1b-454c-8731-74c9d65eaa5a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.770300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:53.792381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb339e1-584b-4dd8-9a84-1bd7004f8ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.793034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb339e1-584b-4dd8-9a84-1bd7004f8ffc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.793310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb339e1-584b-4dd8-9a84-1bd7004f8ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.793585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb339e1-584b-4dd8-9a84-1bd7004f8ffc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.864737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb339e1-584b-4dd8-9a84-1bd7004f8ffc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:53.866204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb339e1-584b-4dd8-9a84-1bd7004f8ffc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:53.917167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:53.930332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616027b3-68f3-48d9-8fe4-c717f5617782 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:53.966168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:53.967079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:53.967373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:53.967654 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/b5d0f422d7c944f5a52ae581e2eb18af/roles/70a666d4a1c5476790ac8f65c8a167f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:53.969898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b3bdd01c0a249b5a7dfce37407324f1, user_id=b5d0f422d7c944f5a52ae581e2eb18af, role_id=70a666d4a1c5476790ac8f65c8a167f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:53.971966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:53.988286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 147/291] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:12:53 2022] PUT /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/users/b5d0f422d7c944f5a52ae581e2eb18af/roles/70a666d4a1c5476790ac8f65c8a167f3 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:54.044540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 145/292] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:12:54.051069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:54.198854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe2cfb1-d1de-4563-b13c-c735299da747 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:54.227331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2721242-2d90-441a-8aaa-6ef7a1b6afce None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.227761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2721242-2d90-441a-8aaa-6ef7a1b6afce None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.228228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2721242-2d90-441a-8aaa-6ef7a1b6afce None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.232987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 146/293] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:54.238533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923014e4-2156-4daf-831c-565926e34108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:54.238772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129e41a-8f2a-4b3e-baae-7442e0374eea None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.239011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129e41a-8f2a-4b3e-baae-7442e0374eea None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.239283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c129e41a-8f2a-4b3e-baae-7442e0374eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.239384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923014e4-2156-4daf-831c-565926e34108 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.239673 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923014e4-2156-4daf-831c-565926e34108 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.239994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923014e4-2156-4daf-831c-565926e34108 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.241198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923014e4-2156-4daf-831c-565926e34108 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:54.242644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923014e4-2156-4daf-831c-565926e34108 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:54.244642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 148/294] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:54 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:54.254623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923014e4-2156-4daf-831c-565926e34108 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:54.265395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c129e41a-8f2a-4b3e-baae-7442e0374eea None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:54.271100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c129e41a-8f2a-4b3e-baae-7442e0374eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:54.289384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:54.316140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:54.399762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 147/295] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:54.421973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923014e4-2156-4daf-831c-565926e34108 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:54.432633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129e41a-8f2a-4b3e-baae-7442e0374eea None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:54.475962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a60e0e-d090-428a-bc5b-86df31165a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:54.475962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a60e0e-d090-428a-bc5b-86df31165a98 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.476271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a60e0e-d090-428a-bc5b-86df31165a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.476538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a60e0e-d090-428a-bc5b-86df31165a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.477369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a60e0e-d090-428a-bc5b-86df31165a98 None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=08d09c5c92af4c5aab225c91131cd6e0) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:54.477683 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a60e0e-d090-428a-bc5b-86df31165a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:54.479111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a60e0e-d090-428a-bc5b-86df31165a98 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:54.485932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 149/296] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:54 2022] GET /identity/v3/role_assignments?user.id=08d09c5c92af4c5aab225c91131cd6e0&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 608 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:54.487188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0106877-a4c5-424e-9d55-672869b46423 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.487498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0106877-a4c5-424e-9d55-672869b46423 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.487769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0106877-a4c5-424e-9d55-672869b46423 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.493020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 150/297] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:54.496278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81d9a5-0334-40f2-9a07-49c266c236df None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.496881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81d9a5-0334-40f2-9a07-49c266c236df None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.497313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a81d9a5-0334-40f2-9a07-49c266c236df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.519219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0a81d9a5-0334-40f2-9a07-49c266c236df None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:54.520823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a81d9a5-0334-40f2-9a07-49c266c236df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:54.580819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c129e41a-8f2a-4b3e-baae-7442e0374eea None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:54.621570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:54.622435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.622721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.623001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.624266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:54.625747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:54.627546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 151/298] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:54.632122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 148/299] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:54.633086 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:54.634150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40712129-64a5-4716-97af-45c3665abaad None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.634651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40712129-64a5-4716-97af-45c3665abaad None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.635143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40712129-64a5-4716-97af-45c3665abaad None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.642220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 152/300] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:54 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:54.650781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130be81-5ce6-4b12-b47f-5b81814ad1e0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.651116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130be81-5ce6-4b12-b47f-5b81814ad1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.651346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130be81-5ce6-4b12-b47f-5b81814ad1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.664441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:54.673351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7130be81-5ce6-4b12-b47f-5b81814ad1e0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:54.674610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7130be81-5ce6-4b12-b47f-5b81814ad1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:54.676801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:54.754726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 153/301] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:54.759965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130be81-5ce6-4b12-b47f-5b81814ad1e0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:54.775319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9811488e-922a-44e0-9645-983486dd63cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:54.822244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:54.823328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.823609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.824058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] PATH_INFO: `/v3/groups` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.824748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:54.826200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:54.842095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 149/302] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Jul 7 10:12:54 2022] POST /identity/v3/groups => generated 229 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:54.842891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:54.892082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130be81-5ce6-4b12-b47f-5b81814ad1e0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:54.937459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:54.950189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:54.952008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:54.952438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:54.952854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:54.963695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:54.973316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:54.976986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b42fe-0ce6-457b-9c27-52a36ac99eab None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:54.983911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:54.985902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:54.987497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:54.990317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 154/303] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:54.996846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:55.030534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:55.031342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:55.031634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:55.031933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:55.045500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:55.053486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:55.054372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:55.055149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 150/304] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:55.060637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:55.122352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5585f0b4-c106-470b-8561-4f706087b513 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:55.167157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:55.167392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145db884-941c-4d89-bc32-6f1b35dee58e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:55.167866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:55.168215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:55.168502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:55.170456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:55.170781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:55.172195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:55.183766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 155/305] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:55.190418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:55.203719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:55.210621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:55.211298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:55.211704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:55.212018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:55.212775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:55.213117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:55.214686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:55.224425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 151/306] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:55.230214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:55.272502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:55.335444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67014f2a-12bc-46fd-a213-afef8e846027 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:55.396743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb862d72-0103-44f6-aa79-1e30cfefb0c9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:55.402723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:55.404485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:55.405252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:55.405997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] PATH_INFO: `/v3/users/201b8995892c44d4a715a9e8da374a2d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:55.434393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] RBAC: Authorizing `identity:get_user(user_id=201b8995892c44d4a715a9e8da374a2d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:55.438451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:55.443015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 156/307] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:55 2022] GET /identity/v3/users/201b8995892c44d4a715a9e8da374a2d => generated 293 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:55.452180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:55.454184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:55.455956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:55.456966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:55.458591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] PATH_INFO: `/v3/users/f0ac3957137e4a35bb1b362c881ff4aa` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:55.460842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] RBAC: Authorizing `identity:get_user(user_id=f0ac3957137e4a35bb1b362c881ff4aa)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:55.463580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:55.467525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 152/308] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:55 2022] GET /identity/v3/users/f0ac3957137e4a35bb1b362c881ff4aa => generated 293 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:55.474210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:55.546418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:55.586061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09981113-d60d-4aff-bfeb-0fe02966363b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:55.591369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a927a9c0-1549-41de-bb96-512fcdf2d863 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:55.606765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:55.628616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d590f-44bc-4781-bef7-513ddfdc352c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:55.629278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d590f-44bc-4781-bef7-513ddfdc352c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:55.629560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d590f-44bc-4781-bef7-513ddfdc352c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:55.629835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d590f-44bc-4781-bef7-513ddfdc352c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:55.630667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d590f-44bc-4781-bef7-513ddfdc352c None admin] RBAC: Adding query filter params (role.id=70a666d4a1c5476790ac8f65c8a167f3, user.id=201b8995892c44d4a715a9e8da374a2d, scope.system=all) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:55.630982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d590f-44bc-4781-bef7-513ddfdc352c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:55.632460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d590f-44bc-4781-bef7-513ddfdc352c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:55.632801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82de0166-0478-4c74-b16e-162bbfab3aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:55.633991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de0166-0478-4c74-b16e-162bbfab3aec None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:55.634468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de0166-0478-4c74-b16e-162bbfab3aec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:55.635029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82de0166-0478-4c74-b16e-162bbfab3aec None admin] PATH_INFO: `/v3/system/users/f0ac3957137e4a35bb1b362c881ff4aa/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:55.636926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de0166-0478-4c74-b16e-162bbfab3aec None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=f0ac3957137e4a35bb1b362c881ff4aa, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:55.643783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 157/309] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jul 7 10:12:55 2022] GET /identity/v3/role_assignments?user.id=201b8995892c44d4a715a9e8da374a2d&scope.system=all&role.id=70a666d4a1c5476790ac8f65c8a167f3 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:55.643993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82de0166-0478-4c74-b16e-162bbfab3aec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:55.653892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 153/310] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Jul 7 10:12:55 2022] PUT /identity/v3/system/users/f0ac3957137e4a35bb1b362c881ff4aa/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:56.229530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb8627-3ff9-4d7c-9430-68a60f6ec075 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:56.230046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb8627-3ff9-4d7c-9430-68a60f6ec075 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:56.230150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb8627-3ff9-4d7c-9430-68a60f6ec075 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:56.231791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 158/311] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:56.237485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1053-76b9-4c42-bad7-3aefe5aa757a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:56.237891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1053-76b9-4c42-bad7-3aefe5aa757a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:56.238078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c1053-76b9-4c42-bad7-3aefe5aa757a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:56.255262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f73c1053-76b9-4c42-bad7-3aefe5aa757a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:56.256485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f73c1053-76b9-4c42-bad7-3aefe5aa757a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:56.323082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 154/312] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:12:56.346523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb8627-3ff9-4d7c-9430-68a60f6ec075 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:56.456677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb8627-3ff9-4d7c-9430-68a60f6ec075 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:56.491936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:56.493078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:56.493560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:56.494090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:56.505744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:56.513994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:56.514849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:56.515635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:56.518568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 159/313] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:12:56.524882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1053-76b9-4c42-bad7-3aefe5aa757a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:56.588408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:56.628640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c1053-76b9-4c42-bad7-3aefe5aa757a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:56.663361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:56.664070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:56.664352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:56.664627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:56.665376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:56.665685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:56.667083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:56.676090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 155/314] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:56.681689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:56.727124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:56.786315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09163d9b-cdfd-4986-9920-4cd596deed2d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:56.823924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce80b17-139a-453e-baa0-2c98ee6b467f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:56.824887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce80b17-139a-453e-baa0-2c98ee6b467f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:56.825172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce80b17-139a-453e-baa0-2c98ee6b467f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:56.825451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce80b17-139a-453e-baa0-2c98ee6b467f None admin] PATH_INFO: `/v3/users/b5d0f422d7c944f5a52ae581e2eb18af` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:56.826805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce80b17-139a-453e-baa0-2c98ee6b467f None admin] RBAC: Authorizing `identity:get_user(user_id=b5d0f422d7c944f5a52ae581e2eb18af)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:56.828351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce80b17-139a-453e-baa0-2c98ee6b467f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:56.830537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 160/315] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:56 2022] GET /identity/v3/users/b5d0f422d7c944f5a52ae581e2eb18af => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:56.836425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:56.941849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7527ff-0966-4076-b183-87f221f6a7a9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:56.968093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:56.976813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:56.977470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:56.977751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:56.978025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.023708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a354b-eabf-47fa-8de2-7cb7bcfe359f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.024031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a354b-eabf-47fa-8de2-7cb7bcfe359f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.024301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949a354b-eabf-47fa-8de2-7cb7bcfe359f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.025842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 161/316] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:57.030733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1bca2-5019-4bb3-b840-d359e327cbc5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.031024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1bca2-5019-4bb3-b840-d359e327cbc5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.031310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe1bca2-5019-4bb3-b840-d359e327cbc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.048316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4fe1bca2-5019-4bb3-b840-d359e327cbc5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:57.050017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4fe1bca2-5019-4bb3-b840-d359e327cbc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:57.051496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:57.053079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:57.124064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 162/317] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:57.140574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1bca2-5019-4bb3-b840-d359e327cbc5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:57.170130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 156/318] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:12:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:12:57.175948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:57.194964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:57.210366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:57.267250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe1bca2-5019-4bb3-b840-d359e327cbc5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:57.298125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc1b78c-793e-41cc-81f9-d79f1c3ebf28 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:57.303820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:57.304579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.304865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.305146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.306342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:57.307827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:57.309647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 163/319] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:12:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:12:57.316302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:57.340394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:57.341421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.341707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.341985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.343225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:57.344750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:57.347076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 157/320] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:12:57 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:57.352749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:57.445225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:57.447371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7e15a9-0ba4-4fa7-9863-a92156ec06ca None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:57.462333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:57.463535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff20cf43-0427-4104-9dc9-d242806bc902 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:57.496562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dcb62d-9593-4ea2-a95f-3e01cb355ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:57.497608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcb62d-9593-4ea2-a95f-3e01cb355ebe None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.498339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcb62d-9593-4ea2-a95f-3e01cb355ebe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.499022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dcb62d-9593-4ea2-a95f-3e01cb355ebe None admin] PATH_INFO: `/v3/groups` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.499913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcb62d-9593-4ea2-a95f-3e01cb355ebe None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:57.501494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dcb62d-9593-4ea2-a95f-3e01cb355ebe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:57.502592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20753838-1fca-4355-ad94-bb8140744766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:57.503252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20753838-1fca-4355-ad94-bb8140744766 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.503537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20753838-1fca-4355-ad94-bb8140744766 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.503841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20753838-1fca-4355-ad94-bb8140744766 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.504750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20753838-1fca-4355-ad94-bb8140744766 None admin] RBAC: Adding query filter params (role.id=70a666d4a1c5476790ac8f65c8a167f3, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1, user.id=b5d0f422d7c944f5a52ae581e2eb18af) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:57.505067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20753838-1fca-4355-ad94-bb8140744766 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:57.506738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20753838-1fca-4355-ad94-bb8140744766 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:57.523829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 158/321] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:12:57 2022] GET /identity/v3/role_assignments?user.id=b5d0f422d7c944f5a52ae581e2eb18af&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=70a666d4a1c5476790ac8f65c8a167f3 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:57.528603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 164/322] 10.222.0.16 () {60 vars in 1134 bytes} [Thu Jul 7 10:12:57 2022] POST /identity/v3/groups => generated 226 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:12:57.739326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcde3b-2bff-413a-af7f-a3c5b1f45916 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.739727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcde3b-2bff-413a-af7f-a3c5b1f45916 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.739954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffcde3b-2bff-413a-af7f-a3c5b1f45916 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.741481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 159/323] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:12:57.746795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e3dc-1dc6-428c-aea3-185cbed38edb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.747087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e3dc-1dc6-428c-aea3-185cbed38edb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.747365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648e3dc-1dc6-428c-aea3-185cbed38edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.765517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d648e3dc-1dc6-428c-aea3-185cbed38edb None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:57.766695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d648e3dc-1dc6-428c-aea3-185cbed38edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:57.798318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085db0-a4af-42f1-947b-a82dc070b84a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.798605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085db0-a4af-42f1-947b-a82dc070b84a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.798948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2085db0-a4af-42f1-947b-a82dc070b84a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.800632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 160/324] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:12:57.805813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cc273-862a-4f15-a98c-5aad3f63c560 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:57.806111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cc273-862a-4f15-a98c-5aad3f63c560 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:57.806379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3cc273-862a-4f15-a98c-5aad3f63c560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:57.832392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ec3cc273-862a-4f15-a98c-5aad3f63c560 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:12:57.833764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec3cc273-862a-4f15-a98c-5aad3f63c560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:12:57.847471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 165/325] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:57.869376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648e3dc-1dc6-428c-aea3-185cbed38edb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:57.928680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 161/326] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:12:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:12:57.951470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cc273-862a-4f15-a98c-5aad3f63c560 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:57.991359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:57.996498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648e3dc-1dc6-428c-aea3-185cbed38edb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.038406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.039381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.039716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.041215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.053067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:58.055239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:58.065289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:58.066200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:12:58.067225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 166/327] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:12:58.072834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:58.078834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3cc273-862a-4f15-a98c-5aad3f63c560 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.119586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.121093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.121093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.121093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.135910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:12:58.143708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:12:58.144627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:12:58.149876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 162/328] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:12:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:12:58.153998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:58.219732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4f7b0c-ba33-43c4-af2e-1eae6e6dd9ce None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.256398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.257010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.257297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.257576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.258335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:58.258645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:58.260906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:58.273677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 167/329] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:58.279772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:58.308426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:58.327163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f0b5ac-6f5f-4e5f-9d98-0d9cab66abba None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.370442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.371117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.371397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.371669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.372458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:58.372762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:58.374410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:58.385561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 163/330] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:12:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:58.391443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:58.394647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:58.419144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb297dfd-20ec-4850-a419-d106415658f9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.465075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.466046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.466276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.466555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] PATH_INFO: `/v3/users/201b8995892c44d4a715a9e8da374a2d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.467866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] RBAC: Authorizing `identity:get_user(user_id=201b8995892c44d4a715a9e8da374a2d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:58.469401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:58.475765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 168/331] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:58 2022] GET /identity/v3/users/201b8995892c44d4a715a9e8da374a2d => generated 293 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:58.478601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:58.537242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51acac10-98ca-438b-85d2-6345b3c47123 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.583400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:58.599406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110bf385-c27c-4bfd-86e1-38d5c9ebf7c3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.604538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.605448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.605731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.606009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] PATH_INFO: `/v3/users/f0ac3957137e4a35bb1b362c881ff4aa` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.607324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] RBAC: Authorizing `identity:get_user(user_id=f0ac3957137e4a35bb1b362c881ff4aa)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:58.609012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:58.610483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 164/332] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:12:58 2022] GET /identity/v3/users/f0ac3957137e4a35bb1b362c881ff4aa => generated 293 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:12:58.616849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:12:58.643776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d611810e-04fc-40b9-884c-b448bab6c6b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.645285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611810e-04fc-40b9-884c-b448bab6c6b2 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.645797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611810e-04fc-40b9-884c-b448bab6c6b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.646266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d611810e-04fc-40b9-884c-b448bab6c6b2 None admin] PATH_INFO: `/v3/system/users/201b8995892c44d4a715a9e8da374a2d/roles/70a666d4a1c5476790ac8f65c8a167f3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.648190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611810e-04fc-40b9-884c-b448bab6c6b2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=201b8995892c44d4a715a9e8da374a2d, role_id=70a666d4a1c5476790ac8f65c8a167f3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:58.650091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d611810e-04fc-40b9-884c-b448bab6c6b2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:58.676542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 169/333] 10.222.0.16 () {58 vars in 1334 bytes} [Thu Jul 7 10:12:58 2022] PUT /identity/v3/system/users/201b8995892c44d4a715a9e8da374a2d/roles/70a666d4a1c5476790ac8f65c8a167f3 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:12:58.704841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:12:58.749948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e443c4f-c7ef-4349-b3cf-c30dae925382 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:12:58.792038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a58f97c-9fdb-406e-ba33-bd475a77a187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:12:58.792642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f97c-9fdb-406e-ba33-bd475a77a187 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:12:58.792927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f97c-9fdb-406e-ba33-bd475a77a187 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:12:58.793206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a58f97c-9fdb-406e-ba33-bd475a77a187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:12:58.794091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58f97c-9fdb-406e-ba33-bd475a77a187 None admin] RBAC: Adding query filter params (role.id=ffd1ee8cb8334eb6a64041118960f6ee, user.id=f0ac3957137e4a35bb1b362c881ff4aa, scope.system=all) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:12:58.794419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58f97c-9fdb-406e-ba33-bd475a77a187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:12:58.795983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a58f97c-9fdb-406e-ba33-bd475a77a187 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:12:58.815498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 165/334] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jul 7 10:12:58 2022] GET /identity/v3/role_assignments?user.id=f0ac3957137e4a35bb1b362c881ff4aa&scope.system=all&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:00.219660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6939-b601-426d-97c5-75759bf6c849 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:00.220195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6939-b601-426d-97c5-75759bf6c849 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:00.220270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c6939-b601-426d-97c5-75759bf6c849 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:00.221910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 170/335] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:00.228094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4700f-d8fa-40d3-aa39-c749840b8ba8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:00.229042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4700f-d8fa-40d3-aa39-c749840b8ba8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:00.229511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed4700f-d8fa-40d3-aa39-c749840b8ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:00.253824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9ed4700f-d8fa-40d3-aa39-c749840b8ba8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:00.255344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9ed4700f-d8fa-40d3-aa39-c749840b8ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:00.343682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 166/336] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:00.378786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd42dfb-d2ca-459e-b03d-918d13a2ac5b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:00.379425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd42dfb-d2ca-459e-b03d-918d13a2ac5b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:00.379944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd42dfb-d2ca-459e-b03d-918d13a2ac5b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:00.382489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 171/337] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:00.384890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4700f-d8fa-40d3-aa39-c749840b8ba8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:00.386412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294c48c-5430-4971-a72e-4c2b199e7096 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:00.386701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294c48c-5430-4971-a72e-4c2b199e7096 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:00.386984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294c48c-5430-4971-a72e-4c2b199e7096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:00.419217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9294c48c-5430-4971-a72e-4c2b199e7096 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:00.420778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9294c48c-5430-4971-a72e-4c2b199e7096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:00.462989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:00.535362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed4700f-d8fa-40d3-aa39-c749840b8ba8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:00.549431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 172/338] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:00.577912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294c48c-5430-4971-a72e-4c2b199e7096 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:00.616052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:00.616865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:00.617151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:00.617429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:00.638294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:00.651311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:00.652416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:00.653321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 167/339] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:00.659534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:00.676409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:00.721582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294c48c-5430-4971-a72e-4c2b199e7096 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:00.791893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:00.792645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:00.792931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:00.793212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:00.820487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:00.828717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:00.829578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:00.830938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:00.831547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 173/340] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:00.844060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:00.879088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29a8dee-6ee8-43dd-8b8e-0aad4afde7b0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:00.919412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:00.972470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:00.973134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:00.973419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:00.973732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:00.974447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:00.974756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:00.976200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:00.998252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 168/341] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:13:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:01.004365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:01.050995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7242a-36e0-4540-9744-e708d33e902e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:01.099561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:01.100578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:01.101003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:01.101418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:01.102321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:01.102770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:01.104314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:01.118446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 174/342] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:01.124119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:01.129706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:01.249701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25a6ae-b80f-4e00-836c-e015ae2a4b56 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:01.254961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec83e262-bb0c-4f19-b2eb-8861f8a0bbaa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:01.292658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5041df-599f-44da-a1c6-7b08e263cbfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:01.293825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5041df-599f-44da-a1c6-7b08e263cbfd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:01.294568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5041df-599f-44da-a1c6-7b08e263cbfd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:01.295073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5041df-599f-44da-a1c6-7b08e263cbfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:01.333038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed84460-a9d3-48e6-803e-f238ea563cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:01.333709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed84460-a9d3-48e6-803e-f238ea563cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:01.334167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed84460-a9d3-48e6-803e-f238ea563cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:01.334599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed84460-a9d3-48e6-803e-f238ea563cc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:01.382240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5041df-599f-44da-a1c6-7b08e263cbfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:01.384110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5041df-599f-44da-a1c6-7b08e263cbfd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:01.393537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:01.466786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:01.497952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 175/343] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:01.499676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb10aa-0475-4d55-acc2-5148f687195e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:01.500168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb10aa-0475-4d55-acc2-5148f687195e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:01.500605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb10aa-0475-4d55-acc2-5148f687195e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:01.502078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 176/344] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:01.502299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed84460-a9d3-48e6-803e-f238ea563cc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:01.503419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb10aa-0475-4d55-acc2-5148f687195e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:01.504513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed84460-a9d3-48e6-803e-f238ea563cc7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:01.620773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb10aa-0475-4d55-acc2-5148f687195e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:01.624332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:01.681376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 169/345] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 677 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:01.684490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb35c1-4254-478c-bdca-5b4fbde53afc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:01.684978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb35c1-4254-478c-bdca-5b4fbde53afc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:01.685403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bb35c1-4254-478c-bdca-5b4fbde53afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:01.693978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:01.695069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:01.698444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:01.698900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:01.700378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:01.702007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:01.704179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 177/346] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:01 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:01.704390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b6bb35c1-4254-478c-bdca-5b4fbde53afc None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:01.705817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:01.706187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6bb35c1-4254-478c-bdca-5b4fbde53afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:01.794425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 170/347] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:01.795287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb35c1-4254-478c-bdca-5b4fbde53afc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:01.832134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:01.911302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bb35c1-4254-478c-bdca-5b4fbde53afc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:01.950825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:01.951695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:01.952147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:01.952508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:01.963542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:01.965052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:01.966780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 171/348] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:01 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:01.967538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:01.969949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbe87d2-e7c5-4cfc-947a-058a665b5a53 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.031416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.032335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.032612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.032908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.034324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:02.035971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:02.038016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 178/349] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:01 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:02.038804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:02.086656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:02.097733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:02.129247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a01920-a8a0-4adb-a2cc-bc96368a32ef None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.147248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a449d1c-6019-4710-8b89-e6d93b0a7948 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.170482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb31ee0-84af-40e9-87b7-13642ae3f523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.171217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb31ee0-84af-40e9-87b7-13642ae3f523 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.171500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb31ee0-84af-40e9-87b7-13642ae3f523 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.171810 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb31ee0-84af-40e9-87b7-13642ae3f523 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.185971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb31ee0-84af-40e9-87b7-13642ae3f523 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:02.190968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b952f4be-5451-4472-94b2-dc3e09c3c620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.191630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f4be-5451-4472-94b2-dc3e09c3c620 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.191943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f4be-5451-4472-94b2-dc3e09c3c620 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.192216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b952f4be-5451-4472-94b2-dc3e09c3c620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.193060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952f4be-5451-4472-94b2-dc3e09c3c620 None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=9930e234fb15434b881890a315de561e) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:02.193373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952f4be-5451-4472-94b2-dc3e09c3c620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:02.196472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b952f4be-5451-4472-94b2-dc3e09c3c620 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-dfb31ee0-84af-40e9-87b7-13642ae3f523 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:02.196843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:02.197749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:02.198542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 172/350] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:02.200170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f701e-2f77-4e9e-b74b-323446a947a5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.200170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f701e-2f77-4e9e-b74b-323446a947a5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.200170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8f701e-2f77-4e9e-b74b-323446a947a5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.201278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 173/351] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:02.201827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f701e-2f77-4e9e-b74b-323446a947a5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:02.207927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 179/352] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=9930e234fb15434b881890a315de561e&role.id=7673131020174664baa4d10a6f38ce2e => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:02.209674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a5228-6d8b-48db-a080-686dffc96fa3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.210164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a5228-6d8b-48db-a080-686dffc96fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.210636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01a5228-6d8b-48db-a080-686dffc96fa3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.214117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 180/353] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:02.215348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a5228-6d8b-48db-a080-686dffc96fa3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:02.337018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8f701e-2f77-4e9e-b74b-323446a947a5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.375708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01a5228-6d8b-48db-a080-686dffc96fa3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.380057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:02.385686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2071ec53-1f71-43dd-ae7b-de7f0b92ad60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.386276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071ec53-1f71-43dd-ae7b-de7f0b92ad60 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.386584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071ec53-1f71-43dd-ae7b-de7f0b92ad60 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.386874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2071ec53-1f71-43dd-ae7b-de7f0b92ad60 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.388764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071ec53-1f71-43dd-ae7b-de7f0b92ad60 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:02.390444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2071ec53-1f71-43dd-ae7b-de7f0b92ad60 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:02.392865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 174/354] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:02.394225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfdb59-6e1f-4922-89d7-5fe7dfd9c6b0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.394712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfdb59-6e1f-4922-89d7-5fe7dfd9c6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.395163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bfdb59-6e1f-4922-89d7-5fe7dfd9c6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.403137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:02.418276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a0bfdb59-6e1f-4922-89d7-5fe7dfd9c6b0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:02.419735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a0bfdb59-6e1f-4922-89d7-5fe7dfd9c6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:02.421912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13545fd-51d9-43fb-8972-72367eda3103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.422529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13545fd-51d9-43fb-8972-72367eda3103 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.422813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13545fd-51d9-43fb-8972-72367eda3103 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.423089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13545fd-51d9-43fb-8972-72367eda3103 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.424818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13545fd-51d9-43fb-8972-72367eda3103 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:02.424818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13545fd-51d9-43fb-8972-72367eda3103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:02.425603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13545fd-51d9-43fb-8972-72367eda3103 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:02.438816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 181/355] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:02.438816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897771c-9b98-429d-a28a-bfd98d7576e5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.438816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897771c-9b98-429d-a28a-bfd98d7576e5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.438816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6897771c-9b98-429d-a28a-bfd98d7576e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.455473 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6897771c-9b98-429d-a28a-bfd98d7576e5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:02.456774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6897771c-9b98-429d-a28a-bfd98d7576e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:02.519949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 175/356] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:02.521519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfdb59-6e1f-4922-89d7-5fe7dfd9c6b0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:02.556577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 182/357] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:02.559319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897771c-9b98-429d-a28a-bfd98d7576e5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:02.684697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bfdb59-6e1f-4922-89d7-5fe7dfd9c6b0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.686391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:02.693963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6897771c-9b98-429d-a28a-bfd98d7576e5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.726719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:02.734047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.735282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.736328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.736814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.737884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=9930e234fb15434b881890a315de561e) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:02.738308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:02.739323 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.740041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.740151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:02.740327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.740590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.747538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 176/358] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=9930e234fb15434b881890a315de561e&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 260 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:02.748206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:02.826838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:02.828360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:02.892784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd15ee4-fbc3-4316-846a-359901603c5c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:02.949722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a46fc7-84a6-43b9-a9c0-043a852abeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:02.951453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a46fc7-84a6-43b9-a9c0-043a852abeae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.951976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a46fc7-84a6-43b9-a9c0-043a852abeae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.952472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a46fc7-84a6-43b9-a9c0-043a852abeae None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.961461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:02.968861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 183/359] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:02.969885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:02.970039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a46fc7-84a6-43b9-a9c0-043a852abeae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c1a46fc7-84a6-43b9-a9c0-043a852abeae None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:02.983106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:02.984048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:02.984848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:02.985447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 177/360] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:02.986944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253accd-6dad-433d-91ea-06582c0d887a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:02.986944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253accd-6dad-433d-91ea-06582c0d887a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:02.991449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9253accd-6dad-433d-91ea-06582c0d887a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:02.994529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 178/361] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:02 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:02.996151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253accd-6dad-433d-91ea-06582c0d887a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.071467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:03.110197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94754b2f-7e5a-4381-8885-48f63c248d58 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.124688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9253accd-6dad-433d-91ea-06582c0d887a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.157042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557797a2-b822-49b7-91ba-69e648e9b084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.158451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557797a2-b822-49b7-91ba-69e648e9b084 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.158838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557797a2-b822-49b7-91ba-69e648e9b084 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.159205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557797a2-b822-49b7-91ba-69e648e9b084 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.169339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.171165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.171670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.172145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.176022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:03.177371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557797a2-b822-49b7-91ba-69e648e9b084 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:03.178490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:03.180836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 179/362] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:03.181838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-557797a2-b822-49b7-91ba-69e648e9b084 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:03.185690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:03.186615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:03.187478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 184/363] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:03.189848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e3cd-0992-473f-a847-6c33c8e1c107 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.190415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e3cd-0992-473f-a847-6c33c8e1c107 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.190980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d05e3cd-0992-473f-a847-6c33c8e1c107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.212822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0d05e3cd-0992-473f-a847-6c33c8e1c107 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:03.214089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0d05e3cd-0992-473f-a847-6c33c8e1c107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:03.281963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:03.303778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 185/364] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:03.304779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05e3cd-0992-473f-a847-6c33c8e1c107 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.322891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5105a3db-0c88-452e-ac59-930837b90a16 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.354473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:03.374950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.375619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.375919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.376189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.377500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:03.377805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:03.379195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:03.395892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 180/365] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:03.396570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.442357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d05e3cd-0992-473f-a847-6c33c8e1c107 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.491293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.492283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.492728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.493148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.494078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:03.494535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:03.496162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:03.512162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 186/366] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:03.516516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.540860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb098be-6b0e-4cee-9de6-41c2c46eca51 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.580793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:03.586747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.588078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.588627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.589044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.590885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:03.592910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:03.595771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 181/367] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:03.596722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.655778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b789b5cc-19d8-43ef-b888-5e0f9d884940 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.666128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:03.694349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.695200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.695583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.695983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.708030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:03.714400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237508b2-0c10-4be6-bd7d-1ec2f3c83995 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:03.717620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:03.718671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:03.719636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:03.722531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 187/368] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:03.722531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.776703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.777270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.777549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.777880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.850215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8127f4bd-07ae-4b4c-91cd-8f37aa265ab0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:03.855887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:03.857401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:03.861979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:03.894603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:03.895611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:03.896175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:03.896460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] PATH_INFO: `/v3/users/201b8995892c44d4a715a9e8da374a2d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:03.898343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] RBAC: Authorizing `identity:get_user(user_id=201b8995892c44d4a715a9e8da374a2d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:03.900062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:03.902915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 188/369] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/users/201b8995892c44d4a715a9e8da374a2d => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:03.903698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:03.931097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:03.986974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 182/370] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:03.987811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:04.032846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f9f27-e065-4323-a6cf-f74b805eddb7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:04.086323 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:04.086998 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.087286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.087568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.088987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:04.089339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:04.091295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:04.099859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 189/371] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:04.100746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:04.152502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:04.161798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da1d3c3-af58-45a0-88da-2853509f0d72 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:04.225815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701ff4b-569a-4416-894a-db494d5373ae None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:04.249116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:04.249804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.250084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.250360 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.251112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:04.252004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:04.253387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:04.264809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:04.265490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.265777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.266054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.266896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] RBAC: Adding query filter params (role.id=70a666d4a1c5476790ac8f65c8a167f3, user.id=201b8995892c44d4a715a9e8da374a2d, scope.system=all) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:04.267268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:04.268726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:04.269639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 183/372] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:04.270688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:04.282337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 190/373] 10.222.0.16 () {56 vars in 1308 bytes} [Thu Jul 7 10:13:04 2022] GET /identity/v3/role_assignments?user.id=201b8995892c44d4a715a9e8da374a2d&scope.system=all&role.id=70a666d4a1c5476790ac8f65c8a167f3 => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:04.283312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:04.321887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:04.429983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:04.436598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50131d97-792f-41cc-8227-f0689a6ffab8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:04.451980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033af7ca-0ee6-4d74-aecc-fc0af7713e6f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:04.517642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:04.518635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.519018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.519393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.528721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:04.530525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:04.533443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 184/374] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:04 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:04.538917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:04.541465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56448e63-2e43-4a84-a68c-7f42f4d71af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:04.542145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56448e63-2e43-4a84-a68c-7f42f4d71af9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.542432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56448e63-2e43-4a84-a68c-7f42f4d71af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.542710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56448e63-2e43-4a84-a68c-7f42f4d71af9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.658768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:04.668124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56448e63-2e43-4a84-a68c-7f42f4d71af9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:04.669619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56448e63-2e43-4a84-a68c-7f42f4d71af9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:04.677842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec33a5-557d-4ba2-ac02-e5a0b0b71393 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:04.721484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d929a7b-97c7-4371-b6d3-9b7902bd9ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:04.722359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929a7b-97c7-4371-b6d3-9b7902bd9ecd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.722643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929a7b-97c7-4371-b6d3-9b7902bd9ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.722916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d929a7b-97c7-4371-b6d3-9b7902bd9ecd None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.725753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d929a7b-97c7-4371-b6d3-9b7902bd9ecd None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:04.726138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d929a7b-97c7-4371-b6d3-9b7902bd9ecd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:04.731051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 185/375] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:04 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:04.731051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227c94-7db7-47da-a4be-930fa78861f9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.731051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227c94-7db7-47da-a4be-930fa78861f9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.731051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33227c94-7db7-47da-a4be-930fa78861f9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.731556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 186/376] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:04 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:04.737080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33227c94-7db7-47da-a4be-930fa78861f9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:04.808238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:04.818312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 191/377] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:04.818628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd256a5-2264-4922-80ee-f6029ea3a1b4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.818875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd256a5-2264-4922-80ee-f6029ea3a1b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.819154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd256a5-2264-4922-80ee-f6029ea3a1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.844532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1cd256a5-2264-4922-80ee-f6029ea3a1b4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:04.846045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cd256a5-2264-4922-80ee-f6029ea3a1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:04.886224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33227c94-7db7-47da-a4be-930fa78861f9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:04.934996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:04.935710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:04.936135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:04.936312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:04.937148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:04.937456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:04.939808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:04.943658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 192/378] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:04.944730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd256a5-2264-4922-80ee-f6029ea3a1b4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:04.950898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 187/379] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:04 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=7673131020174664baa4d10a6f38ce2e => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:04.968733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.006119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:05.103147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd256a5-2264-4922-80ee-f6029ea3a1b4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.146024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:05.162167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.163064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.163345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.163616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] PATH_INFO: `/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.164985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] RBAC: Authorizing `identity:get_project(project_id=3e93b5f74d6546a8b33441fdeb0e9c0d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:05.166447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:05.170830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 193/380] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:04 2022] GET /identity/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d => generated 344 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:05.175882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.207170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee679032-0763-42fa-ae79-205022f147a7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.261773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ec192-1c6e-440a-bdb1-087c7bbd138e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.262591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ec192-1c6e-440a-bdb1-087c7bbd138e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.262869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ec192-1c6e-440a-bdb1-087c7bbd138e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.263139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ec192-1c6e-440a-bdb1-087c7bbd138e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.279485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0ec192-1c6e-440a-bdb1-087c7bbd138e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cd0ec192-1c6e-440a-bdb1-087c7bbd138e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:05.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:05.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:05.290382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 188/381] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:05.291733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc09c-8fb4-4e99-bcbe-c35ef96fd6a3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.293226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc09c-8fb4-4e99-bcbe-c35ef96fd6a3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.293509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dc09c-8fb4-4e99-bcbe-c35ef96fd6a3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.294798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 189/382] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:05.297281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc09c-8fb4-4e99-bcbe-c35ef96fd6a3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.338896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275ba8e6-dd9c-481b-ac09-5e643dc9581a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.384243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3500a04-4dd4-419b-b42d-09c5e8847c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.385043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3500a04-4dd4-419b-b42d-09c5e8847c8e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.385324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3500a04-4dd4-419b-b42d-09c5e8847c8e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.385599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3500a04-4dd4-419b-b42d-09c5e8847c8e None admin] PATH_INFO: `/v3/groups/77d61b74326a43dcb427d55f8b119f8b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.391520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:05.393226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3500a04-4dd4-419b-b42d-09c5e8847c8e None admin] RBAC: Authorizing `identity:get_group(group_id=77d61b74326a43dcb427d55f8b119f8b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:05.396058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3500a04-4dd4-419b-b42d-09c5e8847c8e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:05.398662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 194/383] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:05 2022] GET /identity/v3/groups/77d61b74326a43dcb427d55f8b119f8b => generated 229 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:05.399836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6da3-de1c-4956-946c-109d48d04924 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.400125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6da3-de1c-4956-946c-109d48d04924 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.400398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689d6da3-de1c-4956-946c-109d48d04924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.418991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-689d6da3-de1c-4956-946c-109d48d04924 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:05.419269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dc09c-8fb4-4e99-bcbe-c35ef96fd6a3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.420201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-689d6da3-de1c-4956-946c-109d48d04924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:05.457315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.458058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.458258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.458533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.459381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:05.459692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:05.461308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:05.471809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 190/384] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:05.472842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.487985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:05.502265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 195/385] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:05.503410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d6da3-de1c-4956-946c-109d48d04924 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.608811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855c715d-8bcf-4c37-850b-c607f9a45103 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.618566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689d6da3-de1c-4956-946c-109d48d04924 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.650472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.651126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.651344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.651622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.652522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] RBAC: Adding query filter params (group.id=77d61b74326a43dcb427d55f8b119f8b, role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3e93b5f74d6546a8b33441fdeb0e9c0d) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:05.652846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:05.654563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:05.661241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.662485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 191/386] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:05 2022] GET /identity/v3/role_assignments?group.id=77d61b74326a43dcb427d55f8b119f8b&scope.project.id=3e93b5f74d6546a8b33441fdeb0e9c0d&role.id=f5019e69971843dfadbbc6be956053fb => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:05.662769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.663128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.663340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.663775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.665158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:05.666226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:05.670114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:05.672226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 196/387] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:05 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:05.679668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.811925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:05.849591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c42131-2a89-4a1f-8fea-a8ed73eee800 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.872067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dbc391-2ec5-4330-b00d-2df4dea50c60 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:05.887142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.888152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.888271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.888510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.900930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:05.909555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:05.910622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:05.911599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 192/388] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:05.917037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.920130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:05.921188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:05.921628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:05.922050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:05.924011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:05.925578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:05.927697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 197/389] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:05 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:05.933728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:05.985017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:06.027270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d501fbb-c7f7-4027-b4fa-841e909f5b7a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:06.046865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d67ef3-4329-4473-a7ee-843d87738af0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:06.063207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:06.064201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:06.064483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:06.064755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:06.065516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:06.065793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:06.067158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:06.075819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:06.076190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 193/390] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:13:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:06.081806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:06.087697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:06.088810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:06.089250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:06.089710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:06.092329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=9930e234fb15434b881890a315de561e, group_id=a8880e802d614de7b6e4bf5e31bbb1d1, role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:06.093978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:06.110513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 198/391] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jul 7 10:13:05 2022] PUT /identity/v3/projects/9930e234fb15434b881890a315de561e/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/7673131020174664baa4d10a6f38ce2e => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:13:06.193336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743e2432-1584-494b-b4de-454f4547b982 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:06.222868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:06.231291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:06.232201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:06.232477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:06.232749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:06.233927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:06.235382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:06.237386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 194/392] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:06 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:06.243021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:06.362955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fb70c-65dd-4b12-b45e-a35ca2d3f5b3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:06.403139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd59b0-7b7a-4027-8bb3-f7990c1ae9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:06.404056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd59b0-7b7a-4027-8bb3-f7990c1ae9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:06.404341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd59b0-7b7a-4027-8bb3-f7990c1ae9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:06.404617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd59b0-7b7a-4027-8bb3-f7990c1ae9e1 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:06.405825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd59b0-7b7a-4027-8bb3-f7990c1ae9e1 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:06.407267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcd59b0-7b7a-4027-8bb3-f7990c1ae9e1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:06.409057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 199/393] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:06 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:06.414734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:06.522417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66cf489-7069-45a2-8d7d-46a51b10a2ca None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:06.557014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d987d9-6714-436e-888f-99b8c12df8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:06.557911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d987d9-6714-436e-888f-99b8c12df8ae None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:06.558192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d987d9-6714-436e-888f-99b8c12df8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:06.558470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d987d9-6714-436e-888f-99b8c12df8ae None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:06.560460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d987d9-6714-436e-888f-99b8c12df8ae None admin] RBAC: Authorizing `identity:create_grant(project_id=9930e234fb15434b881890a315de561e, group_id=a8880e802d614de7b6e4bf5e31bbb1d1, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:06.562043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d987d9-6714-436e-888f-99b8c12df8ae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:06.583174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 195/394] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jul 7 10:13:06 2022] PUT /identity/v3/projects/9930e234fb15434b881890a315de561e/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:06.935495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2728b7-4f61-4fa7-9d47-8a79c21462d6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:06.936537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2728b7-4f61-4fa7-9d47-8a79c21462d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:06.937019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2728b7-4f61-4fa7-9d47-8a79c21462d6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:06.939631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 200/395] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:06.944998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4859e37-3d15-45fc-8f33-6c5320beb16d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:06.945292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4859e37-3d15-45fc-8f33-6c5320beb16d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:06.945571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4859e37-3d15-45fc-8f33-6c5320beb16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:06.964465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a4859e37-3d15-45fc-8f33-6c5320beb16d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:06.965656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a4859e37-3d15-45fc-8f33-6c5320beb16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:07.029444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:07.057931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 196/396] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:07.092250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2728b7-4f61-4fa7-9d47-8a79c21462d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:07.167773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:07.221728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2728b7-4f61-4fa7-9d47-8a79c21462d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:07.265306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371bc3ef-da99-40d2-a6ff-c2935bbbfa22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:07.266163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bc3ef-da99-40d2-a6ff-c2935bbbfa22 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:07.266449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bc3ef-da99-40d2-a6ff-c2935bbbfa22 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:07.267258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371bc3ef-da99-40d2-a6ff-c2935bbbfa22 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:07.279661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371bc3ef-da99-40d2-a6ff-c2935bbbfa22 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-371bc3ef-da99-40d2-a6ff-c2935bbbfa22 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:07.291335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:07.292267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:07.293034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 201/397] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:07.299779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4859e37-3d15-45fc-8f33-6c5320beb16d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:07.429535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c28c5-175e-498b-b38b-00a906bb34b4 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:07.429849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c28c5-175e-498b-b38b-00a906bb34b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:07.430091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1c28c5-175e-498b-b38b-00a906bb34b4 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:07.431461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 202/398] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:07.441139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4859e37-3d15-45fc-8f33-6c5320beb16d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:07.443559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d56b1-98b3-4101-93b4-e7f99ac34cf6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:07.443893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d56b1-98b3-4101-93b4-e7f99ac34cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:07.444177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d56b1-98b3-4101-93b4-e7f99ac34cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:07.474107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8b4d56b1-98b3-4101-93b4-e7f99ac34cf6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:07.475337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8b4d56b1-98b3-4101-93b4-e7f99ac34cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:07.491258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:07.491962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:07.492307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:07.492582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:07.493372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:07.493680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:07.495115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:07.514795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 197/399] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:13:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:07.521046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:07.621606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 203/400] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:07.632717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:07.658912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d56b1-98b3-4101-93b4-e7f99ac34cf6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:07.714982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:07.717429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a365820-8696-4bad-b6be-5c6980da75bd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:07.785417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d56b1-98b3-4101-93b4-e7f99ac34cf6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:07.794861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:07.796064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:07.796557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:07.797033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:07.804461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:07.806154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:07.808325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 198/401] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:07 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:07.814264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:07.822494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:07.823360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:07.823647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:07.823968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:07.834919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:07.847767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:07.848714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:07.849485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 204/402] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:07.855231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:07.963393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2675bf9-5763-4f05-8990-bcdf04b6886e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:07.965829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:07.989483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14f70f1-b351-4e62-b312-d9dca1e9aca8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:08.002160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35a206-6323-4087-9076-f563244977bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.002846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35a206-6323-4087-9076-f563244977bb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.003132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35a206-6323-4087-9076-f563244977bb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.003415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35a206-6323-4087-9076-f563244977bb None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.004213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35a206-6323-4087-9076-f563244977bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:08.004530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35a206-6323-4087-9076-f563244977bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:08.005918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35a206-6323-4087-9076-f563244977bb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:08.013712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:08.014840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 205/403] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:08.020530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35a206-6323-4087-9076-f563244977bb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:08.072479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.073715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.074218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.074684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.076148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:08.077807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:08.079775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 199/404] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:07 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:08.085307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:08.155150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35a206-6323-4087-9076-f563244977bb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:08.199459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.199459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.199459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.199459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.199885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:08.199885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:08.199885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 206/405] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:08 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:08.212714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:08.230069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7210e58-15e6-4f50-a21d-8225967cb98c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:08.274048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:08.275310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.276225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.277781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.277781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.283600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b3bdd01c0a249b5a7dfce37407324f1, group_id=a8880e802d614de7b6e4bf5e31bbb1d1, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:08.285080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:08.311757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 200/406] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jul 7 10:13:08 2022] PUT /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:13:08.348227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05f98f-9605-4d70-8590-7f6f0baf9432 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:08.385205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edf625-661c-4357-9fe4-b538a74263b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.386091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf625-661c-4357-9fe4-b538a74263b1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.386381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf625-661c-4357-9fe4-b538a74263b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.386663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edf625-661c-4357-9fe4-b538a74263b1 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.387896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edf625-661c-4357-9fe4-b538a74263b1 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:08.389356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84edf625-661c-4357-9fe4-b538a74263b1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:08.391357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 207/407] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:08 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:08.396874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:08.471012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf80725-8f49-4bf1-a2fa-442a8acbfe41 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.471330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf80725-8f49-4bf1-a2fa-442a8acbfe41 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.471582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf80725-8f49-4bf1-a2fa-442a8acbfe41 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.473185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 208/408] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:13:08.477657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:08.479102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d640420-af60-4776-a94a-e5425efdb97a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.479439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d640420-af60-4776-a94a-e5425efdb97a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.479961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d640420-af60-4776-a94a-e5425efdb97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.507774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8d640420-af60-4776-a94a-e5425efdb97a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:08.508994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8d640420-af60-4776-a94a-e5425efdb97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:08.560966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1376ff23-3968-4d21-9571-6ab12e417961 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:08.582219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 209/409] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:08.599028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.599938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.600226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.600557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.602949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] RBAC: Authorizing `identity:create_grant(project_id=0b3bdd01c0a249b5a7dfce37407324f1, group_id=a8880e802d614de7b6e4bf5e31bbb1d1, role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:08.604052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d640420-af60-4776-a94a-e5425efdb97a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:08.605337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:08.628942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 201/410] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jul 7 10:13:08 2022] PUT /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1/groups/a8880e802d614de7b6e4bf5e31bbb1d1/roles/7673131020174664baa4d10a6f38ce2e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:08.639056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:08.715117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d640420-af60-4776-a94a-e5425efdb97a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:08.754009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.754837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.755127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.755407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.767319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:08.775277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:08.776211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:08.777068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 210/411] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:08.782761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:08.928156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:08.932493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6350d2eb-e8d2-4a8b-acec-93f14a21d25d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:08.970253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cee3d06-c965-45ca-a82f-722a468c94b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:08.970934 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee3d06-c965-45ca-a82f-722a468c94b7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:08.971216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee3d06-c965-45ca-a82f-722a468c94b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:08.971489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cee3d06-c965-45ca-a82f-722a468c94b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:08.972271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee3d06-c965-45ca-a82f-722a468c94b7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:08.972581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee3d06-c965-45ca-a82f-722a468c94b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:08.973960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cee3d06-c965-45ca-a82f-722a468c94b7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:08.983092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 202/412] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:08.988997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.116829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:09.130978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a3fb0d-370f-4724-9270-5fc580e65804 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:09.139892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153176-7313-4487-8a32-fc6ccb6b0ab0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.140183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153176-7313-4487-8a32-fc6ccb6b0ab0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.140511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4153176-7313-4487-8a32-fc6ccb6b0ab0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.142651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 203/413] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:09.157424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa031189-4124-413c-bc07-7794c937fc6b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.157709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa031189-4124-413c-bc07-7794c937fc6b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.157987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa031189-4124-413c-bc07-7794c937fc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.179178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fa031189-4124-413c-bc07-7794c937fc6b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:09.180322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa031189-4124-413c-bc07-7794c937fc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:09.201149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:09.202017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.202297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.202574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] PATH_INFO: `/v3/groups/77d61b74326a43dcb427d55f8b119f8b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.205539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] RBAC: Authorizing `identity:get_group(group_id=77d61b74326a43dcb427d55f8b119f8b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:09.206995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:09.209640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 211/414] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:08 2022] GET /identity/v3/groups/77d61b74326a43dcb427d55f8b119f8b => generated 229 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:09.215125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.291769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 204/415] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:09.341105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa031189-4124-413c-bc07-7794c937fc6b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.368689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aff917-50b4-46f9-a858-fe170e921939 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:09.411628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd1788-8e1b-4ad9-bcbd-941b2c42950d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:09.412545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1788-8e1b-4ad9-bcbd-941b2c42950d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.412832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1788-8e1b-4ad9-bcbd-941b2c42950d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.413114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd1788-8e1b-4ad9-bcbd-941b2c42950d None admin] PATH_INFO: `/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.415085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd1788-8e1b-4ad9-bcbd-941b2c42950d None admin] RBAC: Authorizing `identity:get_project(project_id=3e93b5f74d6546a8b33441fdeb0e9c0d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:09.417072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd1788-8e1b-4ad9-bcbd-941b2c42950d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:09.419543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 212/416] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:09 2022] GET /identity/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d => generated 344 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:09.421073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcfe99-d0e9-44cf-bf00-ae93479e41f8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.421364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcfe99-d0e9-44cf-bf00-ae93479e41f8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.421646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bcfe99-d0e9-44cf-bf00-ae93479e41f8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.423726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 213/417] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:09.426346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcfe99-d0e9-44cf-bf00-ae93479e41f8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.463272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:09.466952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:09.479481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa031189-4124-413c-bc07-7794c937fc6b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:09.528068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f678837-158a-4ee6-8d69-bd5341debc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:09.529701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f678837-158a-4ee6-8d69-bd5341debc54 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.530127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f678837-158a-4ee6-8d69-bd5341debc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.530553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f678837-158a-4ee6-8d69-bd5341debc54 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.544143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f678837-158a-4ee6-8d69-bd5341debc54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1f678837-158a-4ee6-8d69-bd5341debc54 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:09.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:09.557688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:09.558705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:09.560517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bcfe99-d0e9-44cf-bf00-ae93479e41f8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:09.561291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 205/418] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:09.564488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac032-3528-4d21-a49f-b9adc5373508 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.564617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac032-3528-4d21-a49f-b9adc5373508 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.564857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ac032-3528-4d21-a49f-b9adc5373508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.587035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3c8ac032-3528-4d21-a49f-b9adc5373508 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:09.589074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3c8ac032-3528-4d21-a49f-b9adc5373508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:09.602612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:09.603520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.603868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.604153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] PATH_INFO: `/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d/groups/77d61b74326a43dcb427d55f8b119f8b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.606800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] RBAC: Authorizing `identity:create_grant(project_id=3e93b5f74d6546a8b33441fdeb0e9c0d, group_id=77d61b74326a43dcb427d55f8b119f8b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:09.608453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:09.631775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 214/419] 10.222.0.16 () {58 vars in 1442 bytes} [Thu Jul 7 10:13:09 2022] PUT /identity/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d/groups/77d61b74326a43dcb427d55f8b119f8b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:13:09.631775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.701335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 206/420] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:09.736814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ac032-3528-4d21-a49f-b9adc5373508 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.737548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:09.763015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f485fdd7-90c4-4e0c-a6e5-7b4c4227d9ac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:09.831016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:09.832874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.833533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.835419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.836457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:09.837210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:09.839877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:09.857559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 215/421] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:09.865533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.893552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:09.917803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ac032-3528-4d21-a49f-b9adc5373508 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:09.956361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:09.957541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:09.958053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:09.958550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:09.970141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:09.978372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:09.979259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:09.980078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:09.982815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 207/422] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:09.988684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:09.996040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc988e6b-3c1f-490f-98af-52ebbe1d3fc5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:10.039419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:10.043764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.043764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.043764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.062280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:10.123186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:10.125193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:10.207085 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd42af9-9151-4c9e-9d9e-bece27a3c766 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:10.254136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 216/423] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:13:10.260164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:10.322930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:10.327136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:10.327842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.328166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.328392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.329673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] RBAC: Adding query filter params (name=member) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:10.329981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:10.331358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:10.351234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 208/424] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:13:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:10.356559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:10.369618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:10.387170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b1f6b0-7280-48bf-bf56-c52b9e52c33b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:10.427540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:10.429187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.429764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.430190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.431535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:10.433562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:10.435794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 217/425] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:10 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:10.442937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:10.557958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a48c0be-4377-4ae6-b02c-f38250ccf9fd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:10.580176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:10.601882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:10.603078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.603371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.603652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.604914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:10.606351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:10.608309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 218/426] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:10 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:10.614496 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:10.653893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3419da-66e0-4735-8ca1-6173457a7905 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:10.719431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:10.720393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.721068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.721369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.730281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a58a39-e475-4df4-9051-59071de7c9ec None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:10.768404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22070f2c-0243-42dc-9d12-dbfa0d1d02d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:10.768986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22070f2c-0243-42dc-9d12-dbfa0d1d02d0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.769274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22070f2c-0243-42dc-9d12-dbfa0d1d02d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.769617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22070f2c-0243-42dc-9d12-dbfa0d1d02d0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.770462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22070f2c-0243-42dc-9d12-dbfa0d1d02d0 None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=9930e234fb15434b881890a315de561e) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:10.770874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22070f2c-0243-42dc-9d12-dbfa0d1d02d0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:10.772342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22070f2c-0243-42dc-9d12-dbfa0d1d02d0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:10.778969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 219/427] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:10 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=9930e234fb15434b881890a315de561e&role.id=7673131020174664baa4d10a6f38ce2e => generated 611 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:10.789003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451916-d4e9-4901-b5d2-5ef102acc06d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.789356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451916-d4e9-4901-b5d2-5ef102acc06d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.789644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba451916-d4e9-4901-b5d2-5ef102acc06d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.791441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 220/428] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:10 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:10.804005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47fc8-6984-424e-96d3-4c5d175e7c24 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:10.804308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47fc8-6984-424e-96d3-4c5d175e7c24 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:10.804575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b47fc8-6984-424e-96d3-4c5d175e7c24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:10.810587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:10.812090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:10.816197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:10.835348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d6b47fc8-6984-424e-96d3-4c5d175e7c24 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:10.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d6b47fc8-6984-424e-96d3-4c5d175e7c24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:10.934800 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 209/429] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:13:10.941977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:10.949991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:10.988424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 221/430] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:11.027090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47fc8-6984-424e-96d3-4c5d175e7c24 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.074909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572cb90a-a078-446a-85d0-8bd3bc633c3a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.077590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:11.114496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.115368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.115645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.115942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] PATH_INFO: `/v3/projects/9930e234fb15434b881890a315de561e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.117832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] RBAC: Authorizing `identity:get_project(project_id=9930e234fb15434b881890a315de561e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:11.119336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:11.125440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 210/431] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:10 2022] GET /identity/v3/projects/9930e234fb15434b881890a315de561e => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:11.127529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.166718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b47fc8-6984-424e-96d3-4c5d175e7c24 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.207098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.208344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.208875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.209349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.224632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:11.234508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:11.236070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 07 10:13:11.236854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:11.243912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 222/432] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:11.245750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.246852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d56b3e-54be-4abd-9b4a-c08e709ec399 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.287447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee94f7-f3c5-4faf-a133-3659ca48df14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.288855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee94f7-f3c5-4faf-a133-3659ca48df14 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.289347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee94f7-f3c5-4faf-a133-3659ca48df14 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.289624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee94f7-f3c5-4faf-a133-3659ca48df14 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.291756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee94f7-f3c5-4faf-a133-3659ca48df14 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:11.293284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fee94f7-f3c5-4faf-a133-3659ca48df14 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:11.295091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 211/433] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:11.297517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008accf-bc4e-4818-8356-cf81a8797802 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.297517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008accf-bc4e-4818-8356-cf81a8797802 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.298530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008accf-bc4e-4818-8356-cf81a8797802 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.299958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 212/434] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:11.302846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008accf-bc4e-4818-8356-cf81a8797802 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.304754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:11.351544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:11.367190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a65fc18-8df9-48e4-8418-68dbcebcecf4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.402213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2746d49b-43e4-4a7c-a7da-6f5e5d939987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.402878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d49b-43e4-4a7c-a7da-6f5e5d939987 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.403166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d49b-43e4-4a7c-a7da-6f5e5d939987 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.403438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2746d49b-43e4-4a7c-a7da-6f5e5d939987 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.404211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d49b-43e4-4a7c-a7da-6f5e5d939987 None admin] RBAC: Adding query filter params (name=member) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:11.404519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d49b-43e4-4a7c-a7da-6f5e5d939987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:11.405889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2746d49b-43e4-4a7c-a7da-6f5e5d939987 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:11.414348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 223/435] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:11.415500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983739c8-a891-440e-b209-fdc498476263 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.415811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983739c8-a891-440e-b209-fdc498476263 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.416090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983739c8-a891-440e-b209-fdc498476263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.423895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008accf-bc4e-4818-8356-cf81a8797802 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.432309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-983739c8-a891-440e-b209-fdc498476263 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:11.433495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-983739c8-a891-440e-b209-fdc498476263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:11.462478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.463397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.463900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.464418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.465415 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=9930e234fb15434b881890a315de561e) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:11.465926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:11.467907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:11.474995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 213/436] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=9930e234fb15434b881890a315de561e&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 611 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:11.475677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.518888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 224/437] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:11.544324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983739c8-a891-440e-b209-fdc498476263 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.584602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:11.620776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c059ed0-c512-4f6e-9b9f-04f5fb94eed7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.624565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:11.667521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983739c8-a891-440e-b209-fdc498476263 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.669837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.670490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.670774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.671048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.716085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.717015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.717375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.717722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.736751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:11.746014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:11.747075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:11.747952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 225/438] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:11.755818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.764500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:11.765968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:11.879857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:11.887319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 214/439] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:11.889844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa678667-c187-44e5-b3de-c9d810b02682 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:11.893980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:11.898103 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:11.936596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:11.937278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:11.937557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:11.937833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:11.938890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:11.939198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:11.941814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:11.951848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 226/440] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:11.957689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.020070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73955309-1399-4dd3-8506-1c13ce7e2b97 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.064585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b7ba02-59fc-4eba-8aa9-0010f588d964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.065592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7ba02-59fc-4eba-8aa9-0010f588d964 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.065879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7ba02-59fc-4eba-8aa9-0010f588d964 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.066154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b7ba02-59fc-4eba-8aa9-0010f588d964 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.067343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7ba02-59fc-4eba-8aa9-0010f588d964 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.068812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b7ba02-59fc-4eba-8aa9-0010f588d964 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.071858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 215/441] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:12.073409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f5a0-babf-45a5-850c-f2ac70b7b70b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.073904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f5a0-babf-45a5-850c-f2ac70b7b70b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.074184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f79f5a0-babf-45a5-850c-f2ac70b7b70b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.075684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 216/442] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:12.076266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f5a0-babf-45a5-850c-f2ac70b7b70b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.085031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dbc4a7c-c908-459c-8c1a-3cba60239782 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.122899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.123562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.123880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.124152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.152465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:12.166278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:12.197224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f79f5a0-babf-45a5-850c-f2ac70b7b70b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.202435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.203952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.232146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1389395e-a22f-4b16-9218-b6290dcd5993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.232993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389395e-a22f-4b16-9218-b6290dcd5993 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.233267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389395e-a22f-4b16-9218-b6290dcd5993 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.233543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1389395e-a22f-4b16-9218-b6290dcd5993 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.234698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389395e-a22f-4b16-9218-b6290dcd5993 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.236814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1389395e-a22f-4b16-9218-b6290dcd5993 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.238407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 217/443] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:12.239524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a276ce-9e30-4361-99f4-5365c6cc4480 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.239955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a276ce-9e30-4361-99f4-5365c6cc4480 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.240081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a276ce-9e30-4361-99f4-5365c6cc4480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.255573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-06a276ce-9e30-4361-99f4-5365c6cc4480 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:12.256853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-06a276ce-9e30-4361-99f4-5365c6cc4480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:12.303070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 227/444] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:13:12.304252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.329636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 218/445] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:12.330478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a276ce-9e30-4361-99f4-5365c6cc4480 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.359682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:12.399962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:12.403994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1b1f89-5344-4a71-ae1d-121aa2c3a0b3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.430638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a276ce-9e30-4361-99f4-5365c6cc4480 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.437029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.437686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.437965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.438524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.439349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=ffd1ee8cb8334eb6a64041118960f6ee, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:12.439661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.441150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.446754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 228/446] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=ffd1ee8cb8334eb6a64041118960f6ee => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:12.447514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.474636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.475526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.475839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.476122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] PATH_INFO: `/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.477382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] RBAC: Authorizing `identity:get_project(project_id=0b3bdd01c0a249b5a7dfce37407324f1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.478854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.480668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 219/447] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/projects/0b3bdd01c0a249b5a7dfce37407324f1 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:12.486805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.572172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28eaacd8-6be6-4f0f-9b78-be5dc38dc2f0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.591981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:12.608582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72044de-57d2-45c2-a514-d6aa972aa651 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.611552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.612401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.612689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.612964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.626987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:12.635042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:12.636003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:12.636788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 229/448] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:12.643356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.654846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.655659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.655970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.656246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] PATH_INFO: `/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.657962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] RBAC: Authorizing `identity:get_group(group_id=a8880e802d614de7b6e4bf5e31bbb1d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.659412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.661365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 220/449] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/groups/a8880e802d614de7b6e4bf5e31bbb1d1 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:12.667154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.677757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:12.767861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07adb0a9-bde0-4f30-a664-d036dceeeae2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.791425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450757cf-fd38-49a1-8476-431aa5256da8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.804862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.805543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.805834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.806116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.806873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:12.807183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.808592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.818278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 230/450] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:12.823899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:12.830646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.831302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.831583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.831892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:12.832748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] RBAC: Adding query filter params (group.id=a8880e802d614de7b6e4bf5e31bbb1d1, role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:12.833062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:12.834487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:12.840774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 221/451] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/role_assignments?group.id=a8880e802d614de7b6e4bf5e31bbb1d1&scope.project.id=0b3bdd01c0a249b5a7dfce37407324f1&role.id=7673131020174664baa4d10a6f38ce2e => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:12.866434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:12.927979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b1fb61-ab31-4227-b334-4df7df52043a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:12.963490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:12.964186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:12.964479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:12.964757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:13.034288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:13.035728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:13.075721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:13.134243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 231/452] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:13:13.139999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:13.211167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:13.241590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa5ec82-67c2-46f1-b41b-3d11fa12dede None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:13.273912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:13.274756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:13.275037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:13.275312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] PATH_INFO: `/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:13.276495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] RBAC: Authorizing `identity:get_project(project_id=3e93b5f74d6546a8b33441fdeb0e9c0d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:13.277918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:13.279577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 222/453] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:13:13 2022] GET /identity/v3/projects/3e93b5f74d6546a8b33441fdeb0e9c0d => generated 344 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:13.285639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:13.380143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a281b2-ce38-4357-8828-10fa398f7b9e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:13.413535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180aca5-42a0-4acf-8a9f-2e2d9ac60505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:13.414616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180aca5-42a0-4acf-8a9f-2e2d9ac60505 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:13.414898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180aca5-42a0-4acf-8a9f-2e2d9ac60505 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:13.415168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180aca5-42a0-4acf-8a9f-2e2d9ac60505 None admin] PATH_INFO: `/v3/groups/77d61b74326a43dcb427d55f8b119f8b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:13.416331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180aca5-42a0-4acf-8a9f-2e2d9ac60505 None admin] RBAC: Authorizing `identity:get_group(group_id=77d61b74326a43dcb427d55f8b119f8b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:13.417753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180aca5-42a0-4acf-8a9f-2e2d9ac60505 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:13.419335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 232/454] 10.222.0.16 () {56 vars in 1180 bytes} [Thu Jul 7 10:13:13 2022] GET /identity/v3/groups/77d61b74326a43dcb427d55f8b119f8b => generated 229 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:13.424975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:13.521562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6aa7f7-25dc-4e45-baac-63af982d8e37 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:13.555324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfc00f-e61d-4179-a4e5-55740e38529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:13.555996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc00f-e61d-4179-a4e5-55740e38529b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:13.556274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc00f-e61d-4179-a4e5-55740e38529b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:13.556546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfc00f-e61d-4179-a4e5-55740e38529b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:13.557358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfc00f-e61d-4179-a4e5-55740e38529b None admin] RBAC: Adding query filter params (group.id=77d61b74326a43dcb427d55f8b119f8b, role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3e93b5f74d6546a8b33441fdeb0e9c0d) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:13.557666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfc00f-e61d-4179-a4e5-55740e38529b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:13.559098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfc00f-e61d-4179-a4e5-55740e38529b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:13.564921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 223/455] 10.222.0.16 () {56 vars in 1376 bytes} [Thu Jul 7 10:13:13 2022] GET /identity/v3/role_assignments?group.id=77d61b74326a43dcb427d55f8b119f8b&scope.project.id=3e93b5f74d6546a8b33441fdeb0e9c0d&role.id=f5019e69971843dfadbbc6be956053fb => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:15.885119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785bde-f8a8-4962-8dcf-aa34849b9ec7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:15.886163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785bde-f8a8-4962-8dcf-aa34849b9ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:15.886650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16785bde-f8a8-4962-8dcf-aa34849b9ec7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:15.888500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 233/456] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:15.905252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2f3f-0e9d-494e-8d58-faea0613e626 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:15.906103 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2f3f-0e9d-494e-8d58-faea0613e626 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:15.906574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465b2f3f-0e9d-494e-8d58-faea0613e626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:15.917552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:15.932306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-465b2f3f-0e9d-494e-8d58-faea0613e626 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:15.933516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-465b2f3f-0e9d-494e-8d58-faea0613e626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:16.026056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 224/457] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:16.073244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785bde-f8a8-4962-8dcf-aa34849b9ec7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.089741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:16.153779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dec79a-206b-4ff9-ac40-62e14a790283 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.154373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dec79a-206b-4ff9-ac40-62e14a790283 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.154853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0dec79a-206b-4ff9-ac40-62e14a790283 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.156754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 225/458] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:16.169446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbb7-7b55-46fd-bf24-f1536ae5fb1b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.170036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbb7-7b55-46fd-bf24-f1536ae5fb1b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.170539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4457dbb7-7b55-46fd-bf24-f1536ae5fb1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.200289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4457dbb7-7b55-46fd-bf24-f1536ae5fb1b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:16.201540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4457dbb7-7b55-46fd-bf24-f1536ae5fb1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:16.211219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16785bde-f8a8-4962-8dcf-aa34849b9ec7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:16.250077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:16.251244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.251723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.252249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.263653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:16.265524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:16.285495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:16.286364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:16.287216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:16.287216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:16.287216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:16.287216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:16.287436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 226/459] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:16.287436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f1fc-8dde-423b-89fb-dd401810e431 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.287436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f1fc-8dde-423b-89fb-dd401810e431 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.287623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e19f1fc-8dde-423b-89fb-dd401810e431 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.289703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 234/460] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:16.293155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 227/461] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:16.297969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.299761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6105b-e1a2-45cd-9cee-2675b9087c18 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.300610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6105b-e1a2-45cd-9cee-2675b9087c18 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.302784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba6105b-e1a2-45cd-9cee-2675b9087c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.326640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0ba6105b-e1a2-45cd-9cee-2675b9087c18 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:16.328164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0ba6105b-e1a2-45cd-9cee-2675b9087c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:16.373630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:16.407499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:16.419926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5a4f1b-78f9-40ee-ae17-65f10c843217 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:16.427762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 228/462] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:16.427762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6105b-e1a2-45cd-9cee-2675b9087c18 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.478975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:16.480020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.480500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.481034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.481946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:16.482471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:16.484235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:16.497324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 235/463] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:16.498675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.540230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba6105b-e1a2-45cd-9cee-2675b9087c18 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:16.575812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:16.576484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.576774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.577058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.588220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:16.589683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:16.607619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:16.608544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:16.609359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:16.609359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:16.609359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:16.609359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:16.609359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 229/464] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:16.610104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.611941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3674cca9-3ae4-4986-bab8-6d53c2a1151a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:16.634313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:16.653228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:16.654414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.654896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.655363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.665106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:16.669102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:16.671670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:16.691519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:16.692429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:16.693259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:16.693259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:16.693259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:16.693259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:16.695194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 236/465] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:16.696290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.722837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8963bb8b-dd88-4b4e-bd32-d5f4a57523be None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:16.758464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:16.759139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.759427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.759707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.761001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:16.763518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:16.785154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 230/466] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:13:16 2022] POST /identity/v3/users => generated 246 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:16.786073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.810032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf547eb3-3b35-4178-8a1f-960f163f2a68 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:16.850829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec0ebb7-986c-47d1-b51b-9dbc8adfe11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:16.851906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0ebb7-986c-47d1-b51b-9dbc8adfe11b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.852490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0ebb7-986c-47d1-b51b-9dbc8adfe11b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.852962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec0ebb7-986c-47d1-b51b-9dbc8adfe11b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.853994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0ebb7-986c-47d1-b51b-9dbc8adfe11b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:16.854563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0ebb7-986c-47d1-b51b-9dbc8adfe11b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:16.856440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec0ebb7-986c-47d1-b51b-9dbc8adfe11b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:16.872772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 237/467] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:16.874497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4e440-db4a-4861-9141-0ab2329f31d6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.875031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4e440-db4a-4861-9141-0ab2329f31d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.876099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4e440-db4a-4861-9141-0ab2329f31d6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.877609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 238/468] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:16.880073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4e440-db4a-4861-9141-0ab2329f31d6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:16.906804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:16.910330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da382d2-0184-4624-880b-e75e9c479ae1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:16.921046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:16.952236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5838d5b2-e3a2-4088-9d41-e29d863624eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:16.952923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d5b2-e3a2-4088-9d41-e29d863624eb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.953270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d5b2-e3a2-4088-9d41-e29d863624eb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.953512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5838d5b2-e3a2-4088-9d41-e29d863624eb None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.954255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d5b2-e3a2-4088-9d41-e29d863624eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:16.954565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d5b2-e3a2-4088-9d41-e29d863624eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:16.956019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5838d5b2-e3a2-4088-9d41-e29d863624eb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:16.974739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 231/469] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:16.976159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194bb9e-2342-40e4-b20d-2edb02d89ede None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:16.976453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194bb9e-2342-40e4-b20d-2edb02d89ede None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:16.976734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b194bb9e-2342-40e4-b20d-2edb02d89ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:16.997292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b194bb9e-2342-40e4-b20d-2edb02d89ede None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:16.998497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b194bb9e-2342-40e4-b20d-2edb02d89ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:17.018875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4e440-db4a-4861-9141-0ab2329f31d6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:17.064114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:17.064797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.065093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.065375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.066097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:17.068189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:17.083579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 232/470] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:17.084774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194bb9e-2342-40e4-b20d-2edb02d89ede None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:17.099581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 239/471] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:13:16 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:17.126822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:17.247015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:17.264613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b194bb9e-2342-40e4-b20d-2edb02d89ede None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:17.313788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:17.326156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:17.327368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.327907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.328375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.329805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:17.330719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d2a48a-2ba6-4c87-a1d6-1d4d3d1810bf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:17.331640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:17.359458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 233/472] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:13:17 2022] POST /identity/v3/users => generated 248 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:17.393869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:17.395111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.395396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.395671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.409970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:17.411659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:17.431932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:17.432892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:17.433926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:17.433926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:17.433926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:17.433926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:17.433926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 240/473] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:17.441406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:17.561703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58aeb04f-ffd6-4103-bd60-c7086babc1ec None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:17.600745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f1c153-0a43-42f2-85c7-9fe0f21da3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:17.601416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c153-0a43-42f2-85c7-9fe0f21da3e2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.601757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c153-0a43-42f2-85c7-9fe0f21da3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.601974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f1c153-0a43-42f2-85c7-9fe0f21da3e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.602704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1c153-0a43-42f2-85c7-9fe0f21da3e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:17.603012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1c153-0a43-42f2-85c7-9fe0f21da3e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:17.604436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f1c153-0a43-42f2-85c7-9fe0f21da3e2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:17.617558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 234/474] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:17.622787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:17.633369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc18c4-ee22-4487-920a-d12f5b17f25a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.633661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc18c4-ee22-4487-920a-d12f5b17f25a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.633929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dc18c4-ee22-4487-920a-d12f5b17f25a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.635547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 235/475] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:17.640524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3e178-21d7-465f-b162-c628bdb846d5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.640815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3e178-21d7-465f-b162-c628bdb846d5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.641637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b3e178-21d7-465f-b162-c628bdb846d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.656535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:17.663703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a2b3e178-21d7-465f-b162-c628bdb846d5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:17.666142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2b3e178-21d7-465f-b162-c628bdb846d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:17.748118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a1b45-b158-4660-b5d5-68d70d79b86b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:17.754732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 236/476] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:17.787201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3e178-21d7-465f-b162-c628bdb846d5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:17.788997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:17.789923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.790419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.790900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.791888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:17.793550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:17.804689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:17.821663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 241/477] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:13:17 2022] POST /identity/v3/users => generated 249 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:13:17.947133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b3e178-21d7-465f-b162-c628bdb846d5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:17.994907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:17.995983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:17.996469 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:17.996953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:17.997896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:17.999510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:18.003390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:18.008398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.sql.core [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 07 10:13:18.008398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 07 10:13:18.008398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [parameters: {'id': '6b89d1e8f71641af84f627b1e15b42cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 07 10:13:18.008398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80141) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 07 10:13:18.028902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return ref.to_dict() Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 10:13:18.029824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application yield resource Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.rollback() Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.force_reraise() Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise self.value Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application session.commit() Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 07 10:13:18.030723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._prepare_impl() Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.flush() Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._flush(objects) Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application compat.raise_( Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application flush_context.execute() Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rec.execute(self) Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 07 10:13:18.031621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 10:13:18.032525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 10:13:18.035053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [parameters: {'id': '6b89d1e8f71641af84f627b1e15b42cb', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:18.037127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 07 10:13:18.039245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:18.043170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 237/478] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jul 7 10:13:17 2022] POST /identity/v3/roles => generated 144 bytes in 256 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:18.048615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:18.163777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a608aa73-02da-4db6-bb45-40773ed5f885 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:18.195543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c27bf-f18f-4918-92b1-3bab359c9029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:18.196407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c27bf-f18f-4918-92b1-3bab359c9029 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:18.196695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c27bf-f18f-4918-92b1-3bab359c9029 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:18.196974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c27bf-f18f-4918-92b1-3bab359c9029 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:18.210453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c27bf-f18f-4918-92b1-3bab359c9029 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cc8c27bf-f18f-4918-92b1-3bab359c9029 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:18.218849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:18.219868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:18.220667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 242/479] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:18.225731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:18.368433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156602ed-03ca-4929-96ae-887e2b99a3d5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:18.412755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b6834-0b15-461c-b706-1718bc42c2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:18.413802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6834-0b15-461c-b706-1718bc42c2ff None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:18.414347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6834-0b15-461c-b706-1718bc42c2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:18.414835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b6834-0b15-461c-b706-1718bc42c2ff None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:18.415801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6834-0b15-461c-b706-1718bc42c2ff None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:18.416328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6834-0b15-461c-b706-1718bc42c2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:18.417976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b6834-0b15-461c-b706-1718bc42c2ff None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:18.431152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 238/480] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:19.240744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23c95-5424-40a1-93a2-9978ca9224db None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:19.241264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23c95-5424-40a1-93a2-9978ca9224db None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:19.241264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a23c95-5424-40a1-93a2-9978ca9224db None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:19.242888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 243/481] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:19.248222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32dd41-cd0d-464c-b38b-964bbcdca114 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:19.248612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32dd41-cd0d-464c-b38b-964bbcdca114 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:19.248773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df32dd41-cd0d-464c-b38b-964bbcdca114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:19.272789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-df32dd41-cd0d-464c-b38b-964bbcdca114 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:19.273999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-df32dd41-cd0d-464c-b38b-964bbcdca114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:19.364751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:19.377234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 239/482] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:19.412535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23c95-5424-40a1-93a2-9978ca9224db None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:19.505186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:19.589072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a23c95-5424-40a1-93a2-9978ca9224db None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:19.633644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba3be3e-be5b-4848-8370-8378a40335f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:19.634452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3be3e-be5b-4848-8370-8378a40335f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:19.634740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3be3e-be5b-4848-8370-8378a40335f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:19.635021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba3be3e-be5b-4848-8370-8378a40335f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:19.647285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba3be3e-be5b-4848-8370-8378a40335f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:19.654809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca54d3-07ae-4fea-a57f-f0249f548873 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:19.655093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca54d3-07ae-4fea-a57f-f0249f548873 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1ba3be3e-be5b-4848-8370-8378a40335f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:19.655265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:19.656360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:19.657203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:19.657803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ca54d3-07ae-4fea-a57f-f0249f548873 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:19.657870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 244/484] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:19.657936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 240/484] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:19.665144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb301a1-a59d-4ac4-afcf-44b360820628 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:19.665437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb301a1-a59d-4ac4-afcf-44b360820628 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:19.665723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb301a1-a59d-4ac4-afcf-44b360820628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:19.668658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca54d3-07ae-4fea-a57f-f0249f548873 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:19.683065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bdb301a1-a59d-4ac4-afcf-44b360820628 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:19.684328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bdb301a1-a59d-4ac4-afcf-44b360820628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:19.791815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 245/485] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:19.814465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb301a1-a59d-4ac4-afcf-44b360820628 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:19.853740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ca54d3-07ae-4fea-a57f-f0249f548873 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:19.892724 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:19.918900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:19.920696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:19.921745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:19.922948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:19.926053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:19.926903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:19.928511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:19.938148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 241/486] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:19.944642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:19.981701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb301a1-a59d-4ac4-afcf-44b360820628 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.013818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:20.020375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0cdba-b7cf-435a-b11a-0b762cc7c2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.021185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0cdba-b7cf-435a-b11a-0b762cc7c2a9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.021477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0cdba-b7cf-435a-b11a-0b762cc7c2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.021757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0cdba-b7cf-435a-b11a-0b762cc7c2a9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.033284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0cdba-b7cf-435a-b11a-0b762cc7c2a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d2f0cdba-b7cf-435a-b11a-0b762cc7c2a9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:20.041312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:20.042250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:20.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:20.047329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 246/487] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:20.048704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021bfbe-209f-4be6-8ed1-f254dedc2c21 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.049020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021bfbe-209f-4be6-8ed1-f254dedc2c21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.049284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021bfbe-209f-4be6-8ed1-f254dedc2c21 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.050596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 247/488] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:20.051287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021bfbe-209f-4be6-8ed1-f254dedc2c21 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.075595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5edccb-38b1-461a-a303-034f678d3922 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.121923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc409d9-119c-4f67-bd6f-ea5c05f28a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.122982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc409d9-119c-4f67-bd6f-ea5c05f28a64 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.123283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc409d9-119c-4f67-bd6f-ea5c05f28a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.123550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc409d9-119c-4f67-bd6f-ea5c05f28a64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.137679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc409d9-119c-4f67-bd6f-ea5c05f28a64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.139632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc409d9-119c-4f67-bd6f-ea5c05f28a64 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8cc409d9-119c-4f67-bd6f-ea5c05f28a64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:20.162940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:20.163949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:20.164782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:20.164782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:20.164782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:20.164782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:20.170058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 242/489] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:20.170058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eb8d5-37c6-4d0c-a401-b70774be40df None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.170058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eb8d5-37c6-4d0c-a401-b70774be40df None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.170058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975eb8d5-37c6-4d0c-a401-b70774be40df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.176711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021bfbe-209f-4be6-8ed1-f254dedc2c21 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.184762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:20.197266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-975eb8d5-37c6-4d0c-a401-b70774be40df None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:20.198851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-975eb8d5-37c6-4d0c-a401-b70774be40df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:20.217452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.218147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.218436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.218718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.219590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:20.219938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.221532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.236114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 248/490] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:20.237176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.290897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 243/491] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:20.292510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eb8d5-37c6-4d0c-a401-b70774be40df None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.333498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:20.376583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2cb561-cd9d-4643-8a19-0312747cd32a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.412327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975eb8d5-37c6-4d0c-a401-b70774be40df None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.421189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.421993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.422812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.423191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.424057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:20.424849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.426575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.445239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 249/492] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:20.446382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.456261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.457151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.457505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.457845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.471836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.474059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.477411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:20.499962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:20.501385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:20.502229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:20.502229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:20.502229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:20.502229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:20.502229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 244/493] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:20.507218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.565537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b8c6f6-2e2a-48a4-8ce6-76c3ac7f6bf6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.608894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.610664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.611178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.611675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.620031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:20.624441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:20.633566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:20.634441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:20.635223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:20.635855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da52b97-e2a0-42da-9d11-e8a59fc7a5c2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.641557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 250/494] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:20.641557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.677376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226bae56-c573-4c2f-82c6-bb42404a63f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.678523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bae56-c573-4c2f-82c6-bb42404a63f3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.679053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bae56-c573-4c2f-82c6-bb42404a63f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.679529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226bae56-c573-4c2f-82c6-bb42404a63f3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.688256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bae56-c573-4c2f-82c6-bb42404a63f3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.690168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226bae56-c573-4c2f-82c6-bb42404a63f3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-226bae56-c573-4c2f-82c6-bb42404a63f3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:20.707616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:20.708581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:20.709671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 245/495] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:20.714619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388fb3-231b-4c1b-91aa-7d4d70d3dfb1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.714914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388fb3-231b-4c1b-91aa-7d4d70d3dfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.715194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5388fb3-231b-4c1b-91aa-7d4d70d3dfb1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.720177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 246/496] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:20.720177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388fb3-231b-4c1b-91aa-7d4d70d3dfb1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.733565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:20.748313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01efdd0a-2bcd-4890-b388-4710a6acd248 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.783439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.784142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.784424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.784714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.785419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:20.785716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.787082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.799658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 251/497] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:20.800488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.829181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5388fb3-231b-4c1b-91aa-7d4d70d3dfb1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.864411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac064f-5046-4c4c-bcae-6e5328c9efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.865305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac064f-5046-4c4c-bcae-6e5328c9efdd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.865733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac064f-5046-4c4c-bcae-6e5328c9efdd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.866251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac064f-5046-4c4c-bcae-6e5328c9efdd None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.867152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac064f-5046-4c4c-bcae-6e5328c9efdd None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:20.867605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac064f-5046-4c4c-bcae-6e5328c9efdd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.869160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac064f-5046-4c4c-bcae-6e5328c9efdd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.882158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 247/498] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:20.883300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519798d7-ebd5-48b4-ab7b-ded95a3963cc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.883593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519798d7-ebd5-48b4-ab7b-ded95a3963cc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.883903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519798d7-ebd5-48b4-ab7b-ded95a3963cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.899780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-519798d7-ebd5-48b4-ab7b-ded95a3963cc None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:20.901053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-519798d7-ebd5-48b4-ab7b-ded95a3963cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:20.902665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8207146-2921-4043-a434-c0e3f7b46c95 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:20.904000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:20.938928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:20.939596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:20.939931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:20.940211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:20.941048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:20.941359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:20.942777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:20.972308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 248/499] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:20.973435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519798d7-ebd5-48b4-ab7b-ded95a3963cc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.985379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 252/500] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:20.986704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:20.996774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.085144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519798d7-ebd5-48b4-ab7b-ded95a3963cc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.088101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bacfa0-ba16-457c-89d4-3af7f63229f5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.122322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.122738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.123127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.123410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.123682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.124316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.124772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.125250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.133042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.133550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.134920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.135028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.138038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:21.152349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:21.153241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:21.159317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:21.159317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:21.159317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:21.159317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:21.159535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:21.160365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 07 10:13:21.161233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:21.161730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 253/501] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:21.161795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 249/502] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:20 2022] GET /identity/v3/users/glance => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:21.161866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.161933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.216678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.258168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b28238b-4831-4100-80a8-6c61265d6a66 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.258854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d16306-ce7e-41d4-b59f-5085a5cb907a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.291886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.292379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcad9a39-3b43-4c79-9d2a-86f72c8e9d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.292666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.292948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.293048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad9a39-3b43-4c79-9d2a-86f72c8e9d05 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.293226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.293356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad9a39-3b43-4c79-9d2a-86f72c8e9d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.293520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcad9a39-3b43-4c79-9d2a-86f72c8e9d05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.303088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:21.310864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:21.311893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:21.313030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 250/503] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:21.313681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.364659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad9a39-3b43-4c79-9d2a-86f72c8e9d05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.366046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcad9a39-3b43-4c79-9d2a-86f72c8e9d05 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.373785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.417481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554811e7-051e-41b7-b32c-87ecc9030eaa None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.424715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.452094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.452749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.453033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.453801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.454510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:21.454823 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.456212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.467731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 254/504] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:21.467893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 251/505] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:21.468590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.470024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dbc85-ca6d-4cfc-8bde-9acab96ff836 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.470309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dbc85-ca6d-4cfc-8bde-9acab96ff836 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.470586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7dbc85-ca6d-4cfc-8bde-9acab96ff836 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.473252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 255/506] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:21.473252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7dbc85-ca6d-4cfc-8bde-9acab96ff836 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.576881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0a749f-16f0-4340-9e99-917c1f0811be None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.577924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7dbc85-ca6d-4cfc-8bde-9acab96ff836 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.597087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.611283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.611629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.611981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.612265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.612355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.612538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.612627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.612860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.613278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:21.613583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.613679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:21.613990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.614981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.615408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.622609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 256/507] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:21.623374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.645738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.651602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 252/508] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:21.652454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.726899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84deb838-e8e5-498f-a1ff-8b0b5aec513d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.751590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e64f08-fbc5-479e-af06-09435829ae21 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.760030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d994659-1721-4059-961d-c35cb03eea7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.760997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d994659-1721-4059-961d-c35cb03eea7a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.761105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d994659-1721-4059-961d-c35cb03eea7a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.761367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d994659-1721-4059-961d-c35cb03eea7a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.770101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d994659-1721-4059-961d-c35cb03eea7a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.771537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d994659-1721-4059-961d-c35cb03eea7a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.786989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3d994659-1721-4059-961d-c35cb03eea7a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:21.787295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:21.788284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:13:21.789211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:21.789712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.789712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.789712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.789882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 257/509] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:21.789882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6ce25-8eb2-4ff0-a368-a10bcb9fa5f3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.790009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6ce25-8eb2-4ff0-a368-a10bcb9fa5f3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.790667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6ce25-8eb2-4ff0-a368-a10bcb9fa5f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.800783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.802241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.806521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-29d6ce25-8eb2-4ff0-a368-a10bcb9fa5f3 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:21.807714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-29d6ce25-8eb2-4ff0-a368-a10bcb9fa5f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:21.819667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:21.820658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:21.821526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:21.821526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:21.821526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:21.821526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:21.824014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 253/510] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:21.824685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.853909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.875722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 258/511] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:21.876518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6ce25-8eb2-4ff0-a368-a10bcb9fa5f3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:21.888287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:21.927118 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898ef1d-546d-4e05-ae74-f08a47a586e0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.960047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:21.960836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:21.961121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:21.961407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:21.971778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:21.973227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:21.973330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6ce25-8eb2-4ff0-a368-a10bcb9fa5f3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:21.990882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:21.991733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:21.992603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:21.992603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:21.992603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:21.992603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:21.992603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 254/512] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:21.993226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.010624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.011290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.011579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.011893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.080039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.080902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.081513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.092614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.099731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42385c1-55cf-4e55-9086-1642e97641f5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.133366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.133966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.134248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.134521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.135336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:22.135647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.137100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.153980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 255/513] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:22.155034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.181027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 259/514] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:22.181956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.253959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d56b2-1460-469d-b246-5407accdef8a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.279734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55057749-e846-4c50-8a80-6d759d394731 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.288129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.288786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.289073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.289355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.290066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:22.290376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.290781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.291920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.300755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.308790 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 256/515] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:22.309825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.313974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.314779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.315055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.315339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.325344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.326785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:22.344975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:22.345894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:22.346733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:22.346733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:22.346733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:22.346733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:22.346733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 260/516] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:22.347224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.410465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa86a9-77e1-4163-9cd2-6bb8c73dc3f7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.443994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.444657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.444944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.445221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.445936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:22.446147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cc1564-c139-4e13-8922-899dcb7ecc38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.446591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.447653 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.460835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 257/517] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:22.460835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.484212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.484895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.485181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.485459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.515707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.526051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.555285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.556792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.566320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12ee094-cb3c-408f-8b96-3b27d2f18076 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.599720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.600531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.600809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.601080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.611001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.612482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:22.630068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:22.630901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:22.631757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:22.631757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:22.631757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:22.631757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:22.631757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 258/518] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:22.632408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.656029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 261/519] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:22.656883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.715260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.735973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af39a3e-185f-4eba-a88b-707ac5798054 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.749092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.752793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b366410d-1d60-472f-a006-3edbde5589dd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.771576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.772385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.772666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.772939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.779878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.781348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.786655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.787314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.787601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.787907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.788627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:22.788938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.790299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:22.795459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:22.796446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:22.798011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:22.800025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 259/520] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:22.800765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.806352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 262/521] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:22.806352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.898667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c0f6ac-9f71-42b8-ac60-0da6cc75c10a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.902012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcffd72-52e0-4b6c-a253-5d07bd3db05e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:22.924328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:22.932341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.933130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.933416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.933695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.936169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:22.936823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:22.937105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:22.937434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:22.938662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:22.938970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.940356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.941816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:22.943272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:22.951790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 263/522] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:22.952562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:22.957269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:22.958295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 260/523] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:22.959408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:22.990173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:23.053888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd51c4a-c48d-4560-bfad-080c829b6aa6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.058214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4079d70d-817f-4ac1-a023-daabb4277ed8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.087178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.088001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.088290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.088571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.090694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.091341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.091622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.091933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.092707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:23.093016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.094466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.098373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.112205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 261/524] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:23.112971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:23.117692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:23.118721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:23.119659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:23.119659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:23.119659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:23.119659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:23.119659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 264/525] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:23.120100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.155927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:23.202565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:23.212282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de67ac3c-ad07-433c-99ce-01a49cf68c76 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.221660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a493f9-d066-4730-87cd-7276b655e448 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.246600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.247209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.247488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.247801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.248481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.249950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.255412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.256236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.256519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.256793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.264294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.265747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.268953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 262/526] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:13:23 2022] POST /identity/v3/users => generated 247 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:23.269551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:23.280494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:23.281903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:23.282744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 265/527] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:23.283524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.390475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:23.398681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df4dfc-a51e-4b5f-8912-95781a349650 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.409361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353a612-a6d5-4cdf-bd01-17dcdc56ac78 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.438161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.438828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.439111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.439375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.440242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=0914dada5c70437ba49688789eb31ff9) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:23.440554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.441999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.448695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 266/528] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/role_assignments?user.id=0914dada5c70437ba49688789eb31ff9&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:23.450161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.450288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.450365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.450454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.450699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.451532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:23.451891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.453384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.476294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 263/529] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:23.477302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.487657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:23.606620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f49c83-c6eb-4828-a07e-4ef31310cf3f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.611145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825f2c2e-c260-4922-8a1a-2078070294be None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.649740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.650652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.650768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.651185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.651350 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.651636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.651943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.652047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.652726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:23.653037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.653180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:23.653612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.654477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.655261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.670808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 267/531] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:23.675057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 264/531] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:23.675057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.676198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.700664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:23.759990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:23.793589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc95b3-de3b-4b17-94eb-789a2b0f57ac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.810679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141b8c75-cc34-44c0-bde2-4a463a4ece0c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:23.830248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.830911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.831243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.831471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.832366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=4b320f5e6fc4489484fe6599f4391245) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:23.832682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.834183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:23.848296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 268/532] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/role_assignments?user.id=4b320f5e6fc4489484fe6599f4391245&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:23.849287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:23.854794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:23.855837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:23.856370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:23.857850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:23.940183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:23.942008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:24.016974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:24.052839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 265/533] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:24.062170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:24.064598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:24.107658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5594ff9b-5069-496c-b4ef-023dc9983a69 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:24.156418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:24.157827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:24.158683 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:24.159162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:24.169300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:24.171017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:24.185831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac8ab42-8a5a-41fc-af8f-b255fc479938 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:24.186887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:24.188545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:24.189369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:24.192158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 269/534] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:24.202862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:24.251911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:24.252711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:24.252989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:24.253260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:24.271247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:24.272729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:24.290754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:24.291614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:24.292477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:24.292477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:24.292477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:24.292477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:24.292477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 266/535] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:24.298327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:24.323496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:24.350515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2a950-46c4-4eca-be49-58976c016044 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:24.387626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:24.388336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:24.388627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:24.389051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:24.389854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:24.390170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:24.391641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:24.412320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 270/536] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:24.418360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:24.432060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:24.489321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b323468-bb3d-423d-b476-6a0e6b666967 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:24.558402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:24.559070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:24.559348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:24.559620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:24.560365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:24.560668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:24.562039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:24.585455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 267/537] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:24.592329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:24.613073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7baa17-c33f-4dca-a39c-a35d1f0204fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:24.680538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:24.681633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:24.682128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:24.682611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:24.683624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=cc73785ed9da4ad297b924632b14e92c) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:24.684148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:24.685610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:24.698164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:24.706079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 271/538] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:24 2022] GET /identity/v3/role_assignments?user.id=cc73785ed9da4ad297b924632b14e92c&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 259 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:24.736702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2229b3bc-ae3e-43d1-858f-865b31438075 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:24.797808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:24.799004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:24.799508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:24.800000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:24.807643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:24.809451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:24.826108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:24.826988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:24.827870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:24.831031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 268/539] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:24.837012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:24.948291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:24.989392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a46fa3-ebe2-4be3-ac25-49da8b64d41d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:25.058761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6e11fc-8e40-4533-981a-8375f1f3a78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:25.059435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e11fc-8e40-4533-981a-8375f1f3a78c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:25.059720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e11fc-8e40-4533-981a-8375f1f3a78c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:25.060019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6e11fc-8e40-4533-981a-8375f1f3a78c None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:25.060819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e11fc-8e40-4533-981a-8375f1f3a78c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:25.061133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e11fc-8e40-4533-981a-8375f1f3a78c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:25.063136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6e11fc-8e40-4533-981a-8375f1f3a78c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:25.075846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 272/540] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:25.084967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:25.241096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0b8168-a7dd-4980-9b65-6791cc578db0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:25.276173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48787130-4eb3-4846-869f-38a44dcb9b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:25.277098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787130-4eb3-4846-869f-38a44dcb9b94 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:25.277521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787130-4eb3-4846-869f-38a44dcb9b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:25.277934 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48787130-4eb3-4846-869f-38a44dcb9b94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:25.278895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48787130-4eb3-4846-869f-38a44dcb9b94 None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=61c23b3898c14a99bd1521644345d6ba) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:25.279346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48787130-4eb3-4846-869f-38a44dcb9b94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:25.281016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48787130-4eb3-4846-869f-38a44dcb9b94 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:25.287296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 269/541] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:25 2022] GET /identity/v3/role_assignments?user.id=61c23b3898c14a99bd1521644345d6ba&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:25.599397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646c9e-7450-47d1-a967-6bda55ec9870 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:25.599891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646c9e-7450-47d1-a967-6bda55ec9870 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:25.600176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac646c9e-7450-47d1-a967-6bda55ec9870 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:25.601803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 273/542] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:25.619431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010de280-0062-44b7-8c9b-c74a6273fe26 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:25.619945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010de280-0062-44b7-8c9b-c74a6273fe26 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:25.620019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010de280-0062-44b7-8c9b-c74a6273fe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:25.625095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:25.644829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-010de280-0062-44b7-8c9b-c74a6273fe26 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:25.646057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-010de280-0062-44b7-8c9b-c74a6273fe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:25.739571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 270/543] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:25.782838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac646c9e-7450-47d1-a967-6bda55ec9870 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:25.850026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30d463-79e5-40e1-8e31-d5bac2bf4a53 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:25.850622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30d463-79e5-40e1-8e31-d5bac2bf4a53 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:25.851090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d30d463-79e5-40e1-8e31-d5bac2bf4a53 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:25.852969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 271/544] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:25.858385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095abd12-06f4-47d6-a057-6b7f9b7f70f1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:25.858673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095abd12-06f4-47d6-a057-6b7f9b7f70f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:25.858951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095abd12-06f4-47d6-a057-6b7f9b7f70f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:25.895611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-095abd12-06f4-47d6-a057-6b7f9b7f70f1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:25.896840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-095abd12-06f4-47d6-a057-6b7f9b7f70f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:25.901115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac646c9e-7450-47d1-a967-6bda55ec9870 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:25.938572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:25.939456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:25.939754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:25.940037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:25.950998 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:25.960661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:25.961548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 274/545] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:25.962336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:25.967774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.080608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c61e094-5d75-40ae-906d-fb019cfdc0ea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.127498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 272/546] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:26.149978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095abd12-06f4-47d6-a057-6b7f9b7f70f1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.151602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.152451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.152739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.153141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.153917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:26.154233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:26.155627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:26.176153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 275/547] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:26.180144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.242088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:26.278618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:26.285157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095abd12-06f4-47d6-a057-6b7f9b7f70f1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.320997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.322087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.322566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.323019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.334437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:26.343005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:26.343963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:26.344931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:26.351870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 273/548] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:26.353189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.378382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b59cbb-4b32-44bd-bf99-83253220b629 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.446087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.447056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.447273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.447555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.460583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51638d26-ba1e-43bc-8282-b0631ba3af1a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.469939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:26.471496 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:26.488381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:26.497008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.497855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.498284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.498703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.500116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:26.500594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:26.501790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:26.503347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:26.504589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:26.504589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:26.504589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:26.504589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:26.505264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:26.519122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 274/549] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:26.520662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.523636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 276/550] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:26.528858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.632116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4228bf42-a18a-4bd8-a2e0-00f4720e2729 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.661463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:26.665549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a73ea-b83a-4ed6-a88e-2b266ec6cb19 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.667260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6747a138-f163-4ebc-b540-e6f140598adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.668084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747a138-f163-4ebc-b540-e6f140598adc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.668443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747a138-f163-4ebc-b540-e6f140598adc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.668634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6747a138-f163-4ebc-b540-e6f140598adc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.680020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747a138-f163-4ebc-b540-e6f140598adc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:26.681472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6747a138-f163-4ebc-b540-e6f140598adc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6747a138-f163-4ebc-b540-e6f140598adc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:26.700615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:26.701804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:26.703012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:26.703012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:26.703012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:26.703012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:26.706436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 275/551] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:26.707700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbedb18c-8e35-46d5-8ff7-5068185aa0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.707857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2569f-0a8a-4ad4-afdc-e651416590b0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.708037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2569f-0a8a-4ad4-afdc-e651416590b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.708495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e2569f-0a8a-4ad4-afdc-e651416590b0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.708960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedb18c-8e35-46d5-8ff7-5068185aa0cf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.709411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedb18c-8e35-46d5-8ff7-5068185aa0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.709836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbedb18c-8e35-46d5-8ff7-5068185aa0cf None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.710164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 276/552] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:26.710870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2569f-0a8a-4ad4-afdc-e651416590b0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.711126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedb18c-8e35-46d5-8ff7-5068185aa0cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:26.711585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedb18c-8e35-46d5-8ff7-5068185aa0cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:26.713269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbedb18c-8e35-46d5-8ff7-5068185aa0cf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:26.727317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 277/553] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:26.729063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed43a04-47b3-4b1c-b20c-f67f02bea171 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.729531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed43a04-47b3-4b1c-b20c-f67f02bea171 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.730051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed43a04-47b3-4b1c-b20c-f67f02bea171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.749311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6ed43a04-47b3-4b1c-b20c-f67f02bea171 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:26.751344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6ed43a04-47b3-4b1c-b20c-f67f02bea171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:26.753193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:26.821936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 278/554] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:26.823547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed43a04-47b3-4b1c-b20c-f67f02bea171 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.832211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e2569f-0a8a-4ad4-afdc-e651416590b0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.878323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.878699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.878984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.879262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:26.880035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:26.880353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:26.881734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:26.895121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 277/555] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:26.900061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:26.919247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:26.940406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed43a04-47b3-4b1c-b20c-f67f02bea171 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:26.990482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:26.991919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:26.992221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:26.992501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.001769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.003781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.007056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.008514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7f2a65-a4c1-4520-b6db-00cbc36f718c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:27.026218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:27.027163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:27.028106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:27.030381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 279/556] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:27.031803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.050230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.051035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.051312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.051584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.062189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:27.070163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:27.072159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:27.074672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:27.084373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 278/557] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:27.084373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.149357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184507cd-5412-48ab-973d-8592a9b519f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.189773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f467d8a0-5090-46b6-b9dc-13ee277e741c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.193667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.194490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.194774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.195077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.205282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.207050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.226171 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.226843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:27.227124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:27.229315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:27.232782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:27.237636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.237636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.237636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:27.237636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.237636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.238336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 280/558] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:27.238336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.248174 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.250857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 279/559] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:27.251732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.269511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.352799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d64a0-e052-4989-90f8-c84d9edad6d1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.363092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a2d3ed-b99d-4f95-aa8f-f49c6972cc18 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.389018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab255b41-fd85-4a6e-bd57-131c5cc57bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.389820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255b41-fd85-4a6e-bd57-131c5cc57bee None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.390739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255b41-fd85-4a6e-bd57-131c5cc57bee None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.391114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab255b41-fd85-4a6e-bd57-131c5cc57bee None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.392067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255b41-fd85-4a6e-bd57-131c5cc57bee None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:27.392473 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255b41-fd85-4a6e-bd57-131c5cc57bee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.393953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab255b41-fd85-4a6e-bd57-131c5cc57bee None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.401108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.401716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.402011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.402322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.407661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 281/560] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:27.407844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:27.407844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.408011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60631ebf-cb2e-4de4-a8d3-2b75f8364a36 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.408011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60631ebf-cb2e-4de4-a8d3-2b75f8364a36 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.408011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60631ebf-cb2e-4de4-a8d3-2b75f8364a36 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.408260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.408341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 282/561] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:27.411587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60631ebf-cb2e-4de4-a8d3-2b75f8364a36 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.429602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 280/562] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:27.429602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.489091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.510804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.526277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60631ebf-cb2e-4de4-a8d3-2b75f8364a36 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.537876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841694de-21eb-40f3-9509-803bb97864d9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.559972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71361cbd-fb34-43b9-a5ae-ae869a2a422e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.560624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71361cbd-fb34-43b9-a5ae-ae869a2a422e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.560907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71361cbd-fb34-43b9-a5ae-ae869a2a422e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.561179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71361cbd-fb34-43b9-a5ae-ae869a2a422e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.571929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768d0c2-3a8d-4864-bc13-a8b78632fec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.572712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768d0c2-3a8d-4864-bc13-a8b78632fec2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.572992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768d0c2-3a8d-4864-bc13-a8b78632fec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.573264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768d0c2-3a8d-4864-bc13-a8b78632fec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.584085 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768d0c2-3a8d-4864-bc13-a8b78632fec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.585572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768d0c2-3a8d-4864-bc13-a8b78632fec2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6768d0c2-3a8d-4864-bc13-a8b78632fec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:27.603582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:27.604507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:27.605384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:27.605384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:27.605384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:27.605384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:27.605384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 281/563] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:27.606034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5721e-ccdf-4a1f-8cc5-7eafce0752ef None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.606293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5721e-ccdf-4a1f-8cc5-7eafce0752ef None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.606568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5721e-ccdf-4a1f-8cc5-7eafce0752ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.622338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1ac5721e-ccdf-4a1f-8cc5-7eafce0752ef None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:27.623527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ac5721e-ccdf-4a1f-8cc5-7eafce0752ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:27.631271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71361cbd-fb34-43b9-a5ae-ae869a2a422e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.632751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71361cbd-fb34-43b9-a5ae-ae869a2a422e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.690542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 282/564] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:27.691908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5721e-ccdf-4a1f-8cc5-7eafce0752ef None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.703956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.734172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 283/565] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:27.735490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2781a-c9cb-4446-9236-b5cc23cbb5b7 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.735797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2781a-c9cb-4446-9236-b5cc23cbb5b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.736075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca2781a-c9cb-4446-9236-b5cc23cbb5b7 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.737334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 284/566] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:27.738019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2781a-c9cb-4446-9236-b5cc23cbb5b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.754811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.798233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5721e-ccdf-4a1f-8cc5-7eafce0752ef None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.832743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.833678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.833973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.834250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.841111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca2781a-c9cb-4446-9236-b5cc23cbb5b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.844263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.845692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:27.862775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:27.863667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:27.864589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:27.864589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:27.864589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:27.864589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:27.864589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 283/567] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:27.864876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.880079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.880743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.881029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.881309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:27.882039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:27.882349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:27.883705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:27.895247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 285/568] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:27.896047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:27.927880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.962515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fece9be-756a-480b-b3bc-5b981ebbf6b4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.970532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:27.996012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ed0c7-52f1-47d8-a3d9-42640cf577ed None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:27.996874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860a650-ac87-4181-8df1-ac63a7e03d3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:27.997658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a650-ac87-4181-8df1-ac63a7e03d3a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:27.997938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a650-ac87-4181-8df1-ac63a7e03d3a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:27.998216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860a650-ac87-4181-8df1-ac63a7e03d3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.008142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860a650-ac87-4181-8df1-ac63a7e03d3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d860a650-ac87-4181-8df1-ac63a7e03d3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:28.016768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:28.017944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 284/569] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:28.018964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca572e51-499f-48f2-bac7-6ea4615b4257 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.019900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca572e51-499f-48f2-bac7-6ea4615b4257 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.019900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca572e51-499f-48f2-bac7-6ea4615b4257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.033524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.034321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.034607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.034880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.036110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ca572e51-499f-48f2-bac7-6ea4615b4257 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:28.037292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ca572e51-499f-48f2-bac7-6ea4615b4257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:28.045391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.046865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:28.064329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:28.065339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:28.071117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:28.071117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:28.071117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:28.071117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:28.071117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 286/570] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:28.071117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.106407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 285/571] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:28.107151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca572e51-499f-48f2-bac7-6ea4615b4257 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.152410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.168915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cee312-1fec-418e-8f05-fef0271bbeda None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.201463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.201630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.203108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.203476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.203934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.204685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:28.205018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.206399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.208269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca572e51-499f-48f2-bac7-6ea4615b4257 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.217993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 287/572] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:28.219248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.247326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.248406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.248749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.249080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.256236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.257668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:28.272319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:28.273433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 286/573] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:28.274647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.319210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144e42de-a3f5-4914-a7a4-9a04194b838b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.353697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.354358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.354644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.354921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.355673 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:28.356013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.357379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.365429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 288/574] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:28.366199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.372425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01d3f9-5d00-4c79-8b9f-016089786548 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.377417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.407313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.408007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.408292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.408565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.409309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:28.409620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.411013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.422192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 287/575] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:28.422985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.441251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.469474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123d4456-7847-4aec-8ec2-aa4fdf66a484 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.501897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.502702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.502991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.503275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.513340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:28.521687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:28.522755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 289/576] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:28.523617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.526526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822da5d-178f-41aa-8c7a-f7011b39be24 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.559736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.560554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.560841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.561119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.568739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.570169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:28.584931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:28.585939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:28.586807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 288/577] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:28.587295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.591995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.627400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ecf3c0-cc3a-48e0-a2ed-0bc4457c714c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.660225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.660893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.661177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.661452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.662300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:28.662613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.664075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.664503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.680791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 290/578] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:28.681579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.693172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04689bf8-eb51-49a7-97a5-33c908a2f026 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.727418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.728179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.728463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.728741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.739195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.740721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:28.757953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:28.758862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:28.759760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:28.759760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:28.759760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:28.759760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:28.759760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 289/579] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:28.760056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.783713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7528ef97-7f49-460e-89b2-e810c71ebc8a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.806275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.817280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.818074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.818357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.818631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.825130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.826560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:28.840534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:28.842014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 291/580] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:28.843107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.861406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8089b5e5-af55-4305-8455-8e635027ed66 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.887926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:28.896882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.897543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.897822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.898096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.898823 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:28.899125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.900504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.908890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 290/581] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:28.909874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:28.942137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2abd46-cd69-471c-a2f6-1c56d4157772 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:28.975643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:28.976334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:28.976621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:28.976898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:28.977741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:28.977999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:28.979422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:28.991121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 292/582] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:28.992005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.011178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6444b18-c583-40a1-8b93-bfdd922852f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.036356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.044610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.045402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.045684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.045959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.056258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.057706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:29.074879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:29.075956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:29.081809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:29.081809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:29.081809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:29.081809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:29.081809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 291/583] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:29.081809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.091128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7365f1-7c98-4c61-860f-ecf9b49f2651 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.120654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.126514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.127173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.127458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.127731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.128474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:29.128777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.130120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.141434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 293/584] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:29.142201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.180336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20041e5b-d7ae-4a53-bba7-f08e7ac2bff9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.213595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.214244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.214525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.214797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.215572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:29.215911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.217352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.229153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 292/585] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:29.229857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.244094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93498257-76ea-41a1-8672-8eacd19d82b5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.246518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.278076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.278887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.279155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.279432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.289721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.291163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:29.308177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:29.309121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:29.309991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:29.309991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:29.309991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:29.309991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:29.309991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 294/586] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:29.310270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.330703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12058ad-afa5-4f2b-af4a-e0782bacf173 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.332246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.365359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.366244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.366524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.366797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/0914dada5c70437ba49688789eb31ff9/roles/54eab69c526d4d53807c7d0f9b1862c7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.381100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=0914dada5c70437ba49688789eb31ff9, role_id=54eab69c526d4d53807c7d0f9b1862c7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.382533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.395634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 293/587] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:29 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/0914dada5c70437ba49688789eb31ff9/roles/54eab69c526d4d53807c7d0f9b1862c7 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:29.396447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.411007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56adbef3-7195-47fa-af6d-3490801857f8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.455338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.456100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.456445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.456722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.457525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:29.457910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.460247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.477488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 295/588] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:29.480950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.501297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.520673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e486be-e1d2-43b8-bb0a-d26ded7876d8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.562865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.563671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.563983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.564254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.573272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.574724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:29.589587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:29.590594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:13:29.591609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:29.595333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 294/589] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:29.596191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.602438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ae597-b6af-445a-8a13-f14b2286aa31 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.611105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.642039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.642845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.643129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.643420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.644286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=3e2e080fd7144086ac2cf5628a7a68ed) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:29.644659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.646348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.659967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 296/590] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/role_assignments?user.id=3e2e080fd7144086ac2cf5628a7a68ed&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:29.660961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.706672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7787fde6-1790-474a-a233-97a1c0d331b9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.749478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.751040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.751542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.752143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.753201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:29.753718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.755392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.772025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.773856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 295/591] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:29.774711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.798488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b2868b-a568-4cd4-9055-0a021c8cf3ea None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.843575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.845087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.845535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.845951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.854868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.862107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:29.870764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:29.871658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:29.872582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:29.880293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 297/592] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:29.881503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:29.886832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ec3e8-d330-4485-92dc-4e076067efa9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:29.923065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:29.923771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:29.924054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:29.924328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:29.925175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:29.925486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:29.926938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:29.950301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 296/593] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:29.950301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.014053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61770490-ac31-4000-8e38-ab35397b2b91 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.055021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5c9bca-3a4f-492c-a44a-43a7a2c69de7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.060175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.061275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.061753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.063021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.073086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.074559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.081896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:30.091425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:30.092622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 10:13:30.093780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:30.094940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:30.098902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 298/594] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:30.101402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.104082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.104967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.105411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.105878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.106804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:30.107111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.108641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.126323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 297/595] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:30.127255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.221104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01ca903-043c-4739-ae32-40c6ada446e2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.235521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04b7a00-a420-4884-baf7-e05ba437778f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.260059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.261167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.261732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.262307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.270416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.271213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.271328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.271547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.272492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:30.272802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.274423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.276725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.278385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.295911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 298/596] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:30.295911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:30.301308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:30.302205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:30.303129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:30.303129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:30.303129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:30.303129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:30.306960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 299/597] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:30.307711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.355291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:30.385050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:30.412239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1568683f-a52e-41da-891f-7b0d64ea6d16 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.448833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.449776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.450353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.450467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/4b320f5e6fc4489484fe6599f4391245/roles/54eab69c526d4d53807c7d0f9b1862c7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.455600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd9cb01-5ea6-486a-bfc9-6bf39a7ea38f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.471433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=4b320f5e6fc4489484fe6599f4391245, role_id=54eab69c526d4d53807c7d0f9b1862c7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.473065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.494731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.495956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 299/598] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:30 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/4b320f5e6fc4489484fe6599f4391245/roles/54eab69c526d4d53807c7d0f9b1862c7 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:30.496296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.496816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.497126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.497640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.510668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.512655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:30.533903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:30.534868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:30.535789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:30.535789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:30.535789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:30.535789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:30.538068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 300/599] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:30.543886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.692165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:30.699268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e079ba9-6bfc-44c0-a136-f38b3ceac6d5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.718135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb261f8-5443-43da-ab93-6416339c1535 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.728665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:30.745850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.747240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.748002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.749213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.750680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:30.751380 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.753826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.771142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 300/600] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:30.777854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.787617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.788638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.788932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.789208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.789943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:30.790254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.791621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.804270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 301/601] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:30.809858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.887000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26947543-38b3-4c91-818e-997a78a3a169 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.924409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.925205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.925483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.925768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.930711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ec644-24d3-4ee4-8f3f-2525647ef500 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:30.933577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.935037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:30.950148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:30.951078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:30.952006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 301/602] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:30.957722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:30.975527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:30.976568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:30.977710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:30.978332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:30.978946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:30.986774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:30.988638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:31.004126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:31.005037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:31.005946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:31.008389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 302/603] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:31.014127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:31.042777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:31.067049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2885b4ad-5b53-45f4-ac33-bfe2d552c2fd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:31.103878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:31.104535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:31.104819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:31.105096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:31.105972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:31.106281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:31.107705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:31.121267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 302/604] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:31.126180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36f3f5-db8a-4161-8156-de9461fe167f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:31.127598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:31.161382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:31.162411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:31.162926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:31.163404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:31.164431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:31.164950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:31.166607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:31.180096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 303/605] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:31.186032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:31.214021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:31.240827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344965d3-73e6-4fd2-a716-a00fede308f2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:31.276306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:31.280213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3f85e-a791-4ec3-846b-b0e3974ddc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:31.281742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f85e-a791-4ec3-846b-b0e3974ddc04 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:31.282021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f85e-a791-4ec3-846b-b0e3974ddc04 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:31.282293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3f85e-a791-4ec3-846b-b0e3974ddc04 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/cc73785ed9da4ad297b924632b14e92c/roles/54eab69c526d4d53807c7d0f9b1862c7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:31.295828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983ff763-5e21-48dc-be93-0d6d91e2b89b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:31.299155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f85e-a791-4ec3-846b-b0e3974ddc04 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=cc73785ed9da4ad297b924632b14e92c, role_id=54eab69c526d4d53807c7d0f9b1862c7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:31.300614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3f85e-a791-4ec3-846b-b0e3974ddc04 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:31.320132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 303/606] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:31 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/cc73785ed9da4ad297b924632b14e92c/roles/54eab69c526d4d53807c7d0f9b1862c7 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:31.335266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13c311-90df-48ef-9443-bde5a058aec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:31.336719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c311-90df-48ef-9443-bde5a058aec8 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:31.337310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c311-90df-48ef-9443-bde5a058aec8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:31.337934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13c311-90df-48ef-9443-bde5a058aec8 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/61c23b3898c14a99bd1521644345d6ba/roles/54eab69c526d4d53807c7d0f9b1862c7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:31.355953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13c311-90df-48ef-9443-bde5a058aec8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=61c23b3898c14a99bd1521644345d6ba, role_id=54eab69c526d4d53807c7d0f9b1862c7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:31.357773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13c311-90df-48ef-9443-bde5a058aec8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:31.373396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 304/607] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:31 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/61c23b3898c14a99bd1521644345d6ba/roles/54eab69c526d4d53807c7d0f9b1862c7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:13:31.603967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbfa79-0a07-4881-92d2-7c259f997110 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:31.604316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbfa79-0a07-4881-92d2-7c259f997110 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:31.604530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfbfa79-0a07-4881-92d2-7c259f997110 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:31.605897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 304/608] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:31.611727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e932-08f0-42e4-b14c-25e5b0993b0c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:31.612046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e932-08f0-42e4-b14c-25e5b0993b0c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:31.612330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83e932-08f0-42e4-b14c-25e5b0993b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:31.633919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0c83e932-08f0-42e4-b14c-25e5b0993b0c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:31.635141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c83e932-08f0-42e4-b14c-25e5b0993b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:31.716503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 305/609] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:31.750513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbfa79-0a07-4881-92d2-7c259f997110 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:31.909476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfbfa79-0a07-4881-92d2-7c259f997110 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:31.944915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:31.954786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:31.955601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:31.955911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:31.956191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:31.976676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:31.985136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:31.986064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:31.987330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 305/610] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:31.994203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e932-08f0-42e4-b14c-25e5b0993b0c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:32.078544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:32.128239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83e932-08f0-42e4-b14c-25e5b0993b0c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:32.180311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de6b9e-cedd-4bee-8e51-34da3000fca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:32.180908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de6b9e-cedd-4bee-8e51-34da3000fca9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:32.181193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de6b9e-cedd-4bee-8e51-34da3000fca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:32.181470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de6b9e-cedd-4bee-8e51-34da3000fca9 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:32.182222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de6b9e-cedd-4bee-8e51-34da3000fca9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:32.182533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de6b9e-cedd-4bee-8e51-34da3000fca9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:32.183922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de6b9e-cedd-4bee-8e51-34da3000fca9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:32.195160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 306/611] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:32.201165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:32.222335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfad5b5-4cc2-4635-865b-6c50ca73222a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:32.222335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfad5b5-4cc2-4635-865b-6c50ca73222a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:32.222335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfad5b5-4cc2-4635-865b-6c50ca73222a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:32.222335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 307/612] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:32.227618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c434e9f-fd16-49d4-b8be-686a80edf58e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:32.227945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c434e9f-fd16-49d4-b8be-686a80edf58e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:32.228345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c434e9f-fd16-49d4-b8be-686a80edf58e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:32.262124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7c434e9f-fd16-49d4-b8be-686a80edf58e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:32.263318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7c434e9f-fd16-49d4-b8be-686a80edf58e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:32.397223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ba2e7-d628-4629-af09-5670096249f7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:32.409350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 308/613] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:32.455296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:32.456040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:32.456322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:32.456597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:32.471053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c434e9f-fd16-49d4-b8be-686a80edf58e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:32.485801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:32.487276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:32.534497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:32.535436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:32.536882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:32.536882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:32.536882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:32.536882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:32.536882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 306/614] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:32.543364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:32.586583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:32.625955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:32.671232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c434e9f-fd16-49d4-b8be-686a80edf58e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:32.738091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:32.738822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:32.739105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:32.739376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:32.747960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb25940-82b7-4907-b628-d736468682cb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:32.764551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:32.778738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:32.779770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:32.781199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:32.781816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 309/615] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:32.788882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:32.832511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:32.833884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:32.834420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:32.834888 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:32.835826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:32.837188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:32.839441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:32.865257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 307/616] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:32.872685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:32.928138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e103ea7d-2e47-42a7-b2c6-070d27330f36 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:32.956082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:32.966926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:32.970176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:32.970656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:32.971770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:32.972689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:32.973319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:32.976235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:32.992884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 310/617] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:32.998559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.048067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:33.051086 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334f3d67-3e66-4a77-a3d9-7cb5e65d25f6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.089418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0231d-ec4b-4413-be91-1fee112a4865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.090199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0231d-ec4b-4413-be91-1fee112a4865 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.090477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0231d-ec4b-4413-be91-1fee112a4865 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.090744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0231d-ec4b-4413-be91-1fee112a4865 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.098104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0231d-ec4b-4413-be91-1fee112a4865 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:33.099542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0231d-ec4b-4413-be91-1fee112a4865 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-94b0231d-ec4b-4413-be91-1fee112a4865 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:33.113919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:33.114945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:33.115901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 308/618] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:13:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:33.116568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5f203f-ce87-441c-aafc-52e02aa9d276 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.116775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5834-a6ee-4e6c-864c-90d40c907036 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.116775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5834-a6ee-4e6c-864c-90d40c907036 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.117331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496a5834-a6ee-4e6c-864c-90d40c907036 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.118891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 309/619] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:33.123387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a5834-a6ee-4e6c-864c-90d40c907036 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.160546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71f940-7aa3-4619-a5cc-9cd19bc7b318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.161355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71f940-7aa3-4619-a5cc-9cd19bc7b318 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.161636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71f940-7aa3-4619-a5cc-9cd19bc7b318 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.161913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71f940-7aa3-4619-a5cc-9cd19bc7b318 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.175292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71f940-7aa3-4619-a5cc-9cd19bc7b318 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:33.177441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71f940-7aa3-4619-a5cc-9cd19bc7b318 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7f71f940-7aa3-4619-a5cc-9cd19bc7b318 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:33.200997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:33.202843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:33.204426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:33.204426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:33.204426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:33.204426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:33.207801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 311/620] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:33.209916 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbf614-2815-4713-9268-4eace505bc15 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.210208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbf614-2815-4713-9268-4eace505bc15 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.210487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacbf614-2815-4713-9268-4eace505bc15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.231898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-eacbf614-2815-4713-9268-4eace505bc15 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:33.233818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eacbf614-2815-4713-9268-4eace505bc15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:33.243182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496a5834-a6ee-4e6c-864c-90d40c907036 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.264441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:33.289940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.290617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.290900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.291176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.292061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:33.292503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:33.294340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:33.324203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 310/622] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:33.324203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.324450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 312/622] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:33.329432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbf614-2815-4713-9268-4eace505bc15 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.359654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:33.453538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacbf614-2815-4713-9268-4eace505bc15 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.457848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a44316-13f8-414d-8760-2ac73bfc09a8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.495405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.495732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.496371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.496657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.496935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.497304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.497794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.498275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.499773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:33.500484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:33.502166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:33.515036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 311/623] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:33.516382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.530634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:33.583719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:33.585680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:33.610682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:33.636586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bc1e1e-9cf1-4a8d-874a-859448c43493 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.674274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.675443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.675956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.676504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.688286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:33.696829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:33.697739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:33.698568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:33.701236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 312/624] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:33.702287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.707833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 313/625] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:13:33.709413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.772455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:33.816543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d56b2ec-3dfa-4225-b3c0-25e44d104f89 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.816792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7176c0a-056f-4d5a-8265-09bcdd2fb3fe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:33.849933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.850432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0005d-6656-4d01-8cf6-75512901ac3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:33.850729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.851009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.851129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0005d-6656-4d01-8cf6-75512901ac3f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.851278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.851371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0005d-6656-4d01-8cf6-75512901ac3f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.851563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0005d-6656-4d01-8cf6-75512901ac3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.859528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:33.861011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:33.862092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0005d-6656-4d01-8cf6-75512901ac3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:33.862733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:33.863560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea0005d-6656-4d01-8cf6-75512901ac3f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:33.875939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:33.876884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 313/626] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:33.877820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-2ea0005d-6656-4d01-8cf6-75512901ac3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:33.880548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:33.881384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:33.882246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:33.882246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:33.882246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:33.882246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:33.882246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 314/627] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:33.883434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370008aa-cde0-424a-b651-bf777a5afd82 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.883722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370008aa-cde0-424a-b651-bf777a5afd82 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.884018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370008aa-cde0-424a-b651-bf777a5afd82 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.888208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 315/628] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:33.888208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53922ff1-e3fb-4a32-88a2-b01f224a801a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:33.888208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53922ff1-e3fb-4a32-88a2-b01f224a801a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:33.888208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53922ff1-e3fb-4a32-88a2-b01f224a801a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:33.888546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 316/629] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:33.889288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53922ff1-e3fb-4a32-88a2-b01f224a801a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:33.994280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d46d84-01cb-4cd4-bc1f-d2a0f85a61db None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.007921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53922ff1-e3fb-4a32-88a2-b01f224a801a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.024115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.033758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.034711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.035096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.035475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.036363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:34.036793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.038287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.047551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 314/630] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:34.048538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.052298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b83d7f-9985-47c3-b7fa-f4a8ecd3fc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.053149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b83d7f-9985-47c3-b7fa-f4a8ecd3fc55 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.053436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b83d7f-9985-47c3-b7fa-f4a8ecd3fc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.053710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b83d7f-9985-47c3-b7fa-f4a8ecd3fc55 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.054561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b83d7f-9985-47c3-b7fa-f4a8ecd3fc55 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:34.054873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b83d7f-9985-47c3-b7fa-f4a8ecd3fc55 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.056336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b83d7f-9985-47c3-b7fa-f4a8ecd3fc55 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.074313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 317/631] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jul 7 10:13:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:34.075589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828d9d7-623f-4211-9c57-46de3ba1d6b1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.075965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828d9d7-623f-4211-9c57-46de3ba1d6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.076244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7828d9d7-623f-4211-9c57-46de3ba1d6b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.092257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7828d9d7-623f-4211-9c57-46de3ba1d6b1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:34.093494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7828d9d7-623f-4211-9c57-46de3ba1d6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:34.117197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.159352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c16a8-5217-492e-b38f-b0ff611ef333 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.167777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 318/632] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:34.167777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90530757-3542-48ec-8697-453ea46e0470 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.167777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90530757-3542-48ec-8697-453ea46e0470 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.168078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90530757-3542-48ec-8697-453ea46e0470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.186260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-90530757-3542-48ec-8697-453ea46e0470 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:34.188357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-90530757-3542-48ec-8697-453ea46e0470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:34.205431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.206372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.206850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.207318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.208154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:34.208468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.209849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.224159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 315/633] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:34.224814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.266920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 319/634] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:34.267786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90530757-3542-48ec-8697-453ea46e0470 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.303333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.331674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e731b3fa-dd75-40c6-9ba9-a9b617e408ef None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.351099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.365487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.366281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.366565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.366843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.370953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90530757-3542-48ec-8697-453ea46e0470 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.377317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.378761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:34.396563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:34.397488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:34.398435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:34.398435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:34.398435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:34.398435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:34.398435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 316/635] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:34.398778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.407803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.408615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.408897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.409177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.419075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.420546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:34.438363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:34.439340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:34.440270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:34.440270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:34.440270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:34.440270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:34.440270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 320/636] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:34.440559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.501035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827c64b-9ca4-44fe-aaca-7db40c9e90c1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.520721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.537080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.537881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.538160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.538431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.538900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfa5b6-2951-40f4-91b8-f53761fe6392 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.549390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:34.557204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:34.558310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 317/637] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:34.559152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.576884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.577708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.577993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.578271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.584873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.586298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.588951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:34.600540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:34.601607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 321/638] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:34.602798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.661377 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff189f-ffa6-4abe-b358-d1ab94d44f8a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.694365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.695149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.695430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.695710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.700294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05c4a89-dac2-4683-9a04-6f17725e924f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.705576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:34.713322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:34.714408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 318/639] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:34.715272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.738165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.738899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.739184 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.739457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.740194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:34.740529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.741874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.753097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 322/640] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:34.753863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.756311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.808148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.817186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00aa9f07-3437-4f48-b57d-26aa49180afc None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.850206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.850872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.851155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.851429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.851533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202ed5d5-ebab-403a-af0f-91a75757bacb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:34.852487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.853878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.865434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 319/641] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:34.866207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.884947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.885623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:34.885909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:34.886188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:34.886935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:34.887248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:34.888648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:34.896961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 323/642] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:34.897732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:34.960640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:34.965658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1884c1-da81-456f-9e54-39f61051732e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.998500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c84715-08c0-493c-9d7c-1bf91ea48d2e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:34.999052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:34.999714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.000031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.000310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.001137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:35.001447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.002875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.014431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 320/643] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:35.015761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.029613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.037151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.037833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.038118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.038398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.039149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:35.039458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.040870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.048671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 324/644] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:35.049444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.119034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e0eb24-31fb-4a23-91c4-8c5b91c0a99b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.146750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634761b7-c11b-4d0f-b003-bb2e9bcd7f73 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.151580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.152403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.152687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.152964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.161513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.162966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.171065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:35.176853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:35.177961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 321/645] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:35.179239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.184240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.185033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.185312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.185582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.194081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.195504 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:35.209877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:35.210937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 325/646] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:35.211990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.251593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.282023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb9f929-942a-47fa-88f6-1d7cd115bb5d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.311243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112b55ef-6f59-4bf6-8d2e-0cb957ee5bd2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.314610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.315399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.315686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.315998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.326370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.328404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.344947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:35.345170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:35.346180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:35.351684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:35.351684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:35.351684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:35.351684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:35.351988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.351988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.351988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.352230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 322/647] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:35.352312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=0914dada5c70437ba49688789eb31ff9) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:35.352312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.352475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.352559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.364560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 326/648] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/role_assignments?user.id=0914dada5c70437ba49688789eb31ff9&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:35.364560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.412276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.470144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df48e94-3c07-444d-a62b-1ce68267d01c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.475154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be0029-801d-4c7e-a322-ff760a78af05 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.511452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.512592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.512884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.513166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.514033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:35.514348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.516409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.517831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.519204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.520070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.521410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.535010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 327/649] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:35.535520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.535882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.537195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:35.557262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:35.558234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:35.559114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:35.559114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:35.559114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:35.559114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:35.559114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 323/650] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:35.559402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.648424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66d5cf6-a598-416c-b1d3-984243574670 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.658777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.673816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a513ba-e42f-4c2b-99b0-5f7c8c2ff27e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.685796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.686528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.687572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.687880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.688672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:35.688984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.690412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.707540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 328/651] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:35.707540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.715191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.715893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.716567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.716567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.717221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:35.717531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.718941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.731360 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 324/652] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:35.732760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.773616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.816919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fe1f45-2cbd-4e3f-ad91-c00c6ea763d4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.841748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d5448-4dff-4639-8061-7bcd58fd18bf None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:35.852728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.853387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.853674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.853966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.878092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:35.879316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:35.879603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:35.879912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:35.880627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:35.880932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.882314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.893122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:35.898121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 325/653] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:35.898121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:35.933098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:35.934774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:35.997858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.005245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea660dbc-62c9-4a7e-ae12-d8b5296583ae None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.040741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.041630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.041918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.042198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/3e2e080fd7144086ac2cf5628a7a68ed/roles/54eab69c526d4d53807c7d0f9b1862c7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.047187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 329/654] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:36.048557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.058077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=3e2e080fd7144086ac2cf5628a7a68ed, role_id=54eab69c526d4d53807c7d0f9b1862c7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.059554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.083773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 326/655] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:35 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/3e2e080fd7144086ac2cf5628a7a68ed/roles/54eab69c526d4d53807c7d0f9b1862c7 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:36.083773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.138335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.190827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e574a7fe-b6f3-48ce-8bf4-85d083bf6f32 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.216504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02024cc4-db06-4066-9022-3f3d720829d5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.239983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.240801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.241091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.241369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.249397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.251057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.259039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.260150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.260627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.261077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:36.268337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:36.269307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:36.270257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 330/656] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:36.270876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.273533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.275131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:36.292696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:36.293898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:13:36.294881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:36.297545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 327/657] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/users/cinder => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:36.298581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.307849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.378962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff58f7-4092-4145-b3c9-7ba42ce4e744 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.411279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.424614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.425678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.426804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba508b49-0777-4e92-8c49-71f8371099f8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.427299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.427778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.439054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.440773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:36.459890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:36.460815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:36.461699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:36.461699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:36.461699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:36.461699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:36.464137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 331/658] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:36.465143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.471367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.472757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.472757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.472757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.473569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:36.473887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.475339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.499301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 328/659] 10.222.0.16 () {56 vars in 1139 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:36.500259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.571865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.587836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e454c2-299d-4b0e-8246-074db8f3f275 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.615811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921eb3-8c0b-4ec2-b966-04ca841e26ba None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.624187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.625136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.625624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.626119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.627174 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:36.627686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.629380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.648966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 332/660] 10.222.0.16 () {56 vars in 1137 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:36.650267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.653285 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.653920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.654211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.654492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.656200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:36.656200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.657593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.674562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 329/661] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:36.675449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:36.694247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.756933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d26dd-9e59-4135-98f7-22d069af7b5f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.788728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f0d64b-e6a9-4c97-9226-11f2502082ea None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:36.792231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.792583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.792968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.793373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.815829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.828884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:36.829865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:36.830301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:36.830920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:36.865148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.866765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.906027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:36.906866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:36.909353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:36.974786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 333/662] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:36.974786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.025560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 330/663] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:37.026846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.054220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:37.073856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8400aa19-ef64-47f0-b177-7c7d59bbaab7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.108982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.110075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.110793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.110793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.117935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.119462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.121289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:37.135041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:37.136012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:37.136914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:37.140131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 334/664] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:37.140972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.143759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e6c175-5df7-4d4e-84f8-8030697bbe99 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.181365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.182535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.183025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.183604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.195730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.197594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:37.219511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:37.220499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:37.221382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:37.221382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:37.221382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:37.221382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:37.225538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 331/665] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:37.226461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.247075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4a966f-8221-4166-a3a4-ba2adff81ab5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.284406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.285130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.285474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.285751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.297514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.299009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.316148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:37.317904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:37.319644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:37.321896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:37.321896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:37.321896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:37.321896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:37.323690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 335/666] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:37.324748 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.339960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b9f66d-72cc-4176-941c-44e8df0424db None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.377104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:37.379193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.379912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.380212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.381465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.382265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:37.382588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.384722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.397459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 332/667] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:37.398196 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.436837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8453dcc5-a4a8-4bd8-b257-1faf7bb220ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.472148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.472949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.473341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.473689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.474533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:37.474909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.476400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.489483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 336/668] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:37.490387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.512508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ca710-041c-46fb-b189-c9a98dbeb15e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.552490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526af316-743e-44e9-a9c2-20c26cbde573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.553204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526af316-743e-44e9-a9c2-20c26cbde573 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.553490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526af316-743e-44e9-a9c2-20c26cbde573 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.553754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526af316-743e-44e9-a9c2-20c26cbde573 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.554466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526af316-743e-44e9-a9c2-20c26cbde573 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:37.554767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526af316-743e-44e9-a9c2-20c26cbde573 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.556164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526af316-743e-44e9-a9c2-20c26cbde573 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.558844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:37.576245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 333/669] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:37.576245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a9fe-1f04-486f-869b-61589fa82d1b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.576245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a9fe-1f04-486f-869b-61589fa82d1b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.576245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f369a9fe-1f04-486f-869b-61589fa82d1b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.576245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 334/670] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:37.577404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369a9fe-1f04-486f-869b-61589fa82d1b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.602323 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963cdb46-17e0-4f07-ab83-fcd6510c959b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.614261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:37.641987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.642757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.643165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.643450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.644321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=4b320f5e6fc4489484fe6599f4391245) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:37.644639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.646052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.659947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 337/671] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/role_assignments?user.id=4b320f5e6fc4489484fe6599f4391245&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:37.661521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.689339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f369a9fe-1f04-486f-869b-61589fa82d1b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.729747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87abe43-a96e-414d-ad0d-7133389799b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.731000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87abe43-a96e-414d-ad0d-7133389799b3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.731479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87abe43-a96e-414d-ad0d-7133389799b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.732008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87abe43-a96e-414d-ad0d-7133389799b3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.741437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87abe43-a96e-414d-ad0d-7133389799b3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.743308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87abe43-a96e-414d-ad0d-7133389799b3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f87abe43-a96e-414d-ad0d-7133389799b3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:37.758614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:37.759571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:37.760592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:37.763184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 335/672] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:37.765651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e01341-fb45-4f9b-a25e-7d36aa0dbae0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.766185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e01341-fb45-4f9b-a25e-7d36aa0dbae0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.766675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e01341-fb45-4f9b-a25e-7d36aa0dbae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.785362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b8e01341-fb45-4f9b-a25e-7d36aa0dbae0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:37.786912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b8e01341-fb45-4f9b-a25e-7d36aa0dbae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:37.807240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a90b4e-5ad2-43f4-8eb1-20f8b655bbe6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:37.851043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:37.852267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:37.852765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:37.853098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:37.853254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:37.861202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:37.863154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:37.876427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 336/673] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:37.877288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e01341-fb45-4f9b-a25e-7d36aa0dbae0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:37.885756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:37.886636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:37.887511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 338/674] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:37.891460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:37.934713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.002405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e108b83-7c02-48eb-bf5a-0bbb34d8a224 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.008273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e01341-fb45-4f9b-a25e-7d36aa0dbae0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.038660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.039326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.039610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.039917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.040707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:38.041025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.042452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.046847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.047568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.047887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.048218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.049009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:38.049322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.050905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.056395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 339/675] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:38.057227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.063997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 337/676] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:38.068786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.122328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.161872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.169180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0eb04-680b-47e4-b519-c5e531d12a5b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.184903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb5a6fc-ac8e-4f49-b514-a4f161961c02 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.204779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.205582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.205869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.206147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.218691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:38.222486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fcc8f8-2520-4c57-be29-0bc02ab2a114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.223189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcc8f8-2520-4c57-be29-0bc02ab2a114 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.223480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcc8f8-2520-4c57-be29-0bc02ab2a114 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.223779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fcc8f8-2520-4c57-be29-0bc02ab2a114 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.224686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcc8f8-2520-4c57-be29-0bc02ab2a114 None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=cc73785ed9da4ad297b924632b14e92c) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:38.225004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcc8f8-2520-4c57-be29-0bc02ab2a114 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.226444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fcc8f8-2520-4c57-be29-0bc02ab2a114 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:38.226884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:38.228991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 340/677] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:38.231138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.234041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 338/678] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/role_assignments?user.id=cc73785ed9da4ad297b924632b14e92c&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:38.235176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6a117-38f7-4fc5-870a-3b4f5eaf226b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.235517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6a117-38f7-4fc5-870a-3b4f5eaf226b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.239778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe6a117-38f7-4fc5-870a-3b4f5eaf226b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.239778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 339/679] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:38.239778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6a117-38f7-4fc5-870a-3b4f5eaf226b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.356664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d3b51e-9c16-414b-a226-80f9634ddcf0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.377199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe6a117-38f7-4fc5-870a-3b4f5eaf226b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.390534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.397074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcb6c38-fb0f-4118-b7c2-89362fc28b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.398053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb6c38-fb0f-4118-b7c2-89362fc28b84 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.398339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb6c38-fb0f-4118-b7c2-89362fc28b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.399407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcb6c38-fb0f-4118-b7c2-89362fc28b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.400309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb6c38-fb0f-4118-b7c2-89362fc28b84 None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=61c23b3898c14a99bd1521644345d6ba) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:38.400627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb6c38-fb0f-4118-b7c2-89362fc28b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.402103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcb6c38-fb0f-4118-b7c2-89362fc28b84 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.415980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 341/680] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/role_assignments?user.id=61c23b3898c14a99bd1521644345d6ba&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:38.416637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.417338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.417619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.417893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.418668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:38.418977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.419491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee9a99-3b3d-4422-8b13-a7da7a3179e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.420018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee9a99-3b3d-4422-8b13-a7da7a3179e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.420454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.420779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee9a99-3b3d-4422-8b13-a7da7a3179e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.431509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 340/681] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:38.438147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.442922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-beee9a99-3b3d-4422-8b13-a7da7a3179e1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:38.444572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-beee9a99-3b3d-4422-8b13-a7da7a3179e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:38.462403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.537427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 342/682] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:38.560386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9364ddc-9601-4285-a74d-c7471859dd19 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.563571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee9a99-3b3d-4422-8b13-a7da7a3179e1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.600583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.601657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.601940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.602215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.613900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.615294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:38.638003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:38.638961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:38.640436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:38.640436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:38.640436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:38.640436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:38.640436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 341/683] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:38.645080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.672681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.679547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee9a99-3b3d-4422-8b13-a7da7a3179e1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.716171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.716974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.717260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.717572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.729113 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:38.737541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:38.738483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:38.739507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 343/684] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:38.745470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.760309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.767562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c0cfcb-19f7-48dc-93f9-936a2d76fff2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.809061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.809855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.810144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.810422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.811303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:38.812386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.813758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.828799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 342/685] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:38.834099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.868076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d70ed-444d-44a3-ad2e-44fbe2004030 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:38.910652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:38.912142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:38.912430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:38.912707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:38.913465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:38.913775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:38.915139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:38.929100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 344/686] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:38.929579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:38.935133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:38.955662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010894f1-2b7e-43e5-9e76-7a1cce15a609 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:39.007842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:39.008997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:39.010981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:39.012163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:39.034140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:39.034807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:39.036230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:39.060554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:39.061629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:39.062838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:39.065537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 343/687] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:39.071033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:39.097220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3462dc55-bc50-4b63-9ebd-dbd86b6ee4b0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:39.165649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:39.167114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:39.170285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:39.170775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:39.185854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:39.188697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:39.222948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:39.224127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:39.225241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:39.225241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:39.225241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:39.225241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:39.230899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 345/688] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:39.234131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:39.270643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4477298-9cf2-4972-b30e-e603795c6e38 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:39.306766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:39.307824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:39.308271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:39.308748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:39.309948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:39.310419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:39.312887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:39.337272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 344/689] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:13:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:39.348776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:39.351608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:39.408872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d49510f-a685-4359-91cd-0fd7cd27d6df None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:39.457772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:39.480089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:39.480765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:39.481052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:39.482616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:39.483903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:39.485424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:39.486810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:39.513708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 346/690] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:39.519913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:39.549158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762ebb33-0505-49de-a8d9-8b92df36042f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:39.644459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:39.645129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:39.645411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:39.645684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:39.665755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab1892c-f293-4ed6-82f1-4890aa01eff2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:39.705332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:39.706154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:39.706442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:39.706722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:39.716633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:39.719523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:39.738816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:39.739952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:39.740915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:39.744549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 347/691] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:39.753048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:39.796452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:39.798596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:39.800713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:39.869221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:39.910508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 345/692] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 562 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 07 10:13:39.916646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.018597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c2156-13bd-4efa-85cd-141dff98475d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.025175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224042dc-b3f6-40d0-b8c6-b8d6e820cd54 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.027252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:40.066119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.067464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.067975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.068455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.080552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.082497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.085010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.085850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.085963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.086232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.087081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:40.087381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.088823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:40.103782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:40.104740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:40.105612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:40.105612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:40.105612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:40.105612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:40.107954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 346/693] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:40.113747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.115721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 348/694] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jul 7 10:13:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:40.129223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.230307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861eaa0c-e8e4-4b68-a7d2-ac1a4ee5c37a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.266375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469abd3-8f3f-4b61-9e47-8e6e5a42989d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.267349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:40.267976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469abd3-8f3f-4b61-9e47-8e6e5a42989d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.268691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469abd3-8f3f-4b61-9e47-8e6e5a42989d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.269231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469abd3-8f3f-4b61-9e47-8e6e5a42989d None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.270373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469abd3-8f3f-4b61-9e47-8e6e5a42989d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:40.270884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469abd3-8f3f-4b61-9e47-8e6e5a42989d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.271612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82342707-01e8-4ddb-9fee-275aa61cdec3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.272830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469abd3-8f3f-4b61-9e47-8e6e5a42989d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.291179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 347/695] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:40.292899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab70b5-f5f1-4d69-a96d-126f261624df None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.293395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab70b5-f5f1-4d69-a96d-126f261624df None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.293873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab70b5-f5f1-4d69-a96d-126f261624df None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.296105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 348/696] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:40.296998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab70b5-f5f1-4d69-a96d-126f261624df None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.315774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:40.322385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.323054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.323345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.323626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.415597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab70b5-f5f1-4d69-a96d-126f261624df None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.419858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.421986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.453329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6feab53-79e0-43ab-be09-ebcaf1f738f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.454806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feab53-79e0-43ab-be09-ebcaf1f738f2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.455322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feab53-79e0-43ab-be09-ebcaf1f738f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.455826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6feab53-79e0-43ab-be09-ebcaf1f738f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.463329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feab53-79e0-43ab-be09-ebcaf1f738f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.465139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6feab53-79e0-43ab-be09-ebcaf1f738f2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b6feab53-79e0-43ab-be09-ebcaf1f738f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:40.480736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:40.481677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:40.482580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:40.490537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 349/697] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:40.492117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be0424-f479-47c6-aed1-3bf120153af1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.492877 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be0424-f479-47c6-aed1-3bf120153af1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.493369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66be0424-f479-47c6-aed1-3bf120153af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.514164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-66be0424-f479-47c6-aed1-3bf120153af1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:40.515569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-66be0424-f479-47c6-aed1-3bf120153af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:40.537868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:40.547331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 349/698] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 07 10:13:40.548525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.612193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 350/699] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:40.613434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be0424-f479-47c6-aed1-3bf120153af1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.623470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:40.680495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec20b37c-2da8-4773-9803-e0fb34a8aafd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.715399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.716181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.716465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.716736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.718050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:40.718360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.719916 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.736372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 350/700] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:40.736372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.743231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66be0424-f479-47c6-aed1-3bf120153af1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.779677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555f4f97-8aaf-44e3-9f3a-7d9024ee8f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.781036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f4f97-8aaf-44e3-9f3a-7d9024ee8f57 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.781318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f4f97-8aaf-44e3-9f3a-7d9024ee8f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.781600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555f4f97-8aaf-44e3-9f3a-7d9024ee8f57 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.792443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f4f97-8aaf-44e3-9f3a-7d9024ee8f57 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.794337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555f4f97-8aaf-44e3-9f3a-7d9024ee8f57 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.800209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-555f4f97-8aaf-44e3-9f3a-7d9024ee8f57 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:40.812660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:40.813599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:40.814486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:40.814486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:40.814486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:40.814486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:40.814486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 351/701] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:40.815950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185973-5f1c-4d5d-9a4e-f5ef3b1c11e3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.815950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185973-5f1c-4d5d-9a4e-f5ef3b1c11e3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.815950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f185973-5f1c-4d5d-9a4e-f5ef3b1c11e3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.817344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 352/702] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:40.821793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f185973-5f1c-4d5d-9a4e-f5ef3b1c11e3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.852305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f00333a-be63-461c-bf18-69707fee3be1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.883172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:40.889719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.890819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.891111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.891392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.902817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.904319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:40.922157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:40.923231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:40.929043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:40.929043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:40.929043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:40.929043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:40.929043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 351/703] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:40.929043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:40.931081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f185973-5f1c-4d5d-9a4e-f5ef3b1c11e3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:40.965434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c0a06c-8757-47f0-82d3-6ae3692a4e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:40.966089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0a06c-8757-47f0-82d3-6ae3692a4e73 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.966368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0a06c-8757-47f0-82d3-6ae3692a4e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.966645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0a06c-8757-47f0-82d3-6ae3692a4e73 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:40.968073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0a06c-8757-47f0-82d3-6ae3692a4e73 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=0914dada5c70437ba49688789eb31ff9) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:40.968390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0a06c-8757-47f0-82d3-6ae3692a4e73 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:40.969821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c0a06c-8757-47f0-82d3-6ae3692a4e73 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:40.975295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 353/704] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/role_assignments?user.id=0914dada5c70437ba49688789eb31ff9&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:40.976510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491330f-9dc9-43fb-8964-355c96afe3ea None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:40.976802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491330f-9dc9-43fb-8964-355c96afe3ea None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:40.977071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7491330f-9dc9-43fb-8964-355c96afe3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.006487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7491330f-9dc9-43fb-8964-355c96afe3ea None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:41.007726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7491330f-9dc9-43fb-8964-355c96afe3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:41.027964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763f0201-25bd-4bee-839c-df72140b9387 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.044472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:41.067696 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23010c18-74a7-4499-b5c7-9ba6a11096be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.068407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23010c18-74a7-4499-b5c7-9ba6a11096be None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.068693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23010c18-74a7-4499-b5c7-9ba6a11096be None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.068970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23010c18-74a7-4499-b5c7-9ba6a11096be None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.069701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23010c18-74a7-4499-b5c7-9ba6a11096be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:41.070004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23010c18-74a7-4499-b5c7-9ba6a11096be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.071396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23010c18-74a7-4499-b5c7-9ba6a11096be None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.085071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 352/705] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:41.086875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e2ef9-de44-4266-98ca-d0812c5155d5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.087428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e2ef9-de44-4266-98ca-d0812c5155d5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.088058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e2ef9-de44-4266-98ca-d0812c5155d5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.089879 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 353/706] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:41.090846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e2ef9-de44-4266-98ca-d0812c5155d5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.095818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 354/707] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:41.095818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491330f-9dc9-43fb-8964-355c96afe3ea None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.159634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:41.207533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e2ef9-de44-4266-98ca-d0812c5155d5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.214738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7491330f-9dc9-43fb-8964-355c96afe3ea None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.242286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55d8114-b2eb-450a-bda4-555411ebe032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.242959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d8114-b2eb-450a-bda4-555411ebe032 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.243248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d8114-b2eb-450a-bda4-555411ebe032 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.243531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55d8114-b2eb-450a-bda4-555411ebe032 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.244292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d8114-b2eb-450a-bda4-555411ebe032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:41.244604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d8114-b2eb-450a-bda4-555411ebe032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.245964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55d8114-b2eb-450a-bda4-555411ebe032 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.251392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.252282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.252565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.252865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.262619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 354/708] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:41.262619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddb65d-4a31-4459-88f7-bd5394d8da8f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.262619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddb65d-4a31-4459-88f7-bd5394d8da8f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.262619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ddb65d-4a31-4459-88f7-bd5394d8da8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.265277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.267227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.277533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a0ddb65d-4a31-4459-88f7-bd5394d8da8f None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:41.278735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a0ddb65d-4a31-4459-88f7-bd5394d8da8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:41.281139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:41.282028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:41.282927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 355/709] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:41.283444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.311708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:41.355233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 355/710] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:41.355995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddb65d-4a31-4459-88f7-bd5394d8da8f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.398484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d84fb5f-2f73-46bf-a687-669359927e7b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.409323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:41.434727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.435548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.435861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.436679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.437593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.438972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:41.453860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:41.454840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:41.460489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 356/711] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:41.461363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.474485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ddb65d-4a31-4459-88f7-bd5394d8da8f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.510500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.511185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.511476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.511772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.512495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.514708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.537963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 356/712] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:13:41 2022] POST /identity/v3/users => generated 254 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:41.539161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.568956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:41.596870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2893d351-de1c-425f-8f0c-fcf6b2830279 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.643084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.643939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.644357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.644680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.645650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:41.645961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.648368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.671943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 357/713] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:41.674032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.690481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617adcf9-9560-40fc-97f6-024d5aab8543 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.723798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:41.726637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.727468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.727778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.728063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.728743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.730107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:41.747570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:41.749625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:41.750926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:41.753384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 357/714] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/services/network => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:41.754376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.820089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381d667d-cbe8-4f3e-8ef9-ea6b519de47b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.856469 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.857404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.858357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.858852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.859783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:41.860315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.862010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.867636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 358/715] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:41.868103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:41.868975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:41.885391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff32b9-c772-45ec-b93e-0eb0786af37d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:41.931063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:41.931728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:41.932043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:41.932320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:41.933428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=3e2e080fd7144086ac2cf5628a7a68ed) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:41.934147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:41.935974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:41.944555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 358/716] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/role_assignments?user.id=3e2e080fd7144086ac2cf5628a7a68ed&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:41.945690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:42.022590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:42.032788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5190c112-10bf-4568-bcb9-e653622cf222 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:42.110911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:42.113323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:42.116230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:42.117001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:42.118055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] RBAC: Adding query filter params (name=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:42.118055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:42.122774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:42.124942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 359/717] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:42.131394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:42.147566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb973b95-4d88-4420-910b-89cd291c2ab2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:42.195361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c557d7c-12e3-4cc4-9b99-478a647805a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:42.196091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c557d7c-12e3-4cc4-9b99-478a647805a7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:42.196541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c557d7c-12e3-4cc4-9b99-478a647805a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:42.196818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c557d7c-12e3-4cc4-9b99-478a647805a7 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:42.197826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c557d7c-12e3-4cc4-9b99-478a647805a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:42.198142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c557d7c-12e3-4cc4-9b99-478a647805a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:42.199512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c557d7c-12e3-4cc4-9b99-478a647805a7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:42.207371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 359/718] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jul 7 10:13:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:42.265473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17adb06-a7c3-4710-99e6-cd8b980c2420 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:42.309210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcd4dc9-4590-4eb9-9ce9-d6de5d06d6f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:42.309903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4dc9-4590-4eb9-9ce9-d6de5d06d6f4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:42.310186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4dc9-4590-4eb9-9ce9-d6de5d06d6f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:42.310461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcd4dc9-4590-4eb9-9ce9-d6de5d06d6f4 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:42.311240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd4dc9-4590-4eb9-9ce9-d6de5d06d6f4 None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:42.311549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd4dc9-4590-4eb9-9ce9-d6de5d06d6f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:42.313264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcd4dc9-4590-4eb9-9ce9-d6de5d06d6f4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:42.320974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 360/719] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:13:42 2022] GET /identity/v3/services?type=network => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:43.176821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dea4d-c7d6-4d69-94e0-96e3e8e81feb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:43.177370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dea4d-c7d6-4d69-94e0-96e3e8e81feb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:43.177370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343dea4d-c7d6-4d69-94e0-96e3e8e81feb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:43.178967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 360/720] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:43.184458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9d0-7142-481d-83fc-f98ec5809a70 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:43.184848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9d0-7142-481d-83fc-f98ec5809a70 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:43.185010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5ec9d0-7142-481d-83fc-f98ec5809a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:43.202643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ec5ec9d0-7142-481d-83fc-f98ec5809a70 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:43.203880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec5ec9d0-7142-481d-83fc-f98ec5809a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:43.224614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:43.287522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 361/721] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 07 10:13:43.310079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dea4d-c7d6-4d69-94e0-96e3e8e81feb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:43.324801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:43.434374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343dea4d-c7d6-4d69-94e0-96e3e8e81feb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:43.478385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:43.479093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:43.479386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:43.479670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:43.491866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:43.499672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:43.500581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:43.501399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 361/722] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:43.507112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9d0-7142-481d-83fc-f98ec5809a70 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:43.635109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5ec9d0-7142-481d-83fc-f98ec5809a70 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:43.679306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:43.680308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:43.680729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:43.681165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:43.682053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:43.682518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:43.684078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:43.700972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 362/723] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:43.707223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:43.799608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:43.843323 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8272db9e-6d13-45eb-b9ce-077f1ec7a84e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:43.893810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47293f09-09a3-42a1-bdac-21da8353de71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:43.894617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47293f09-09a3-42a1-bdac-21da8353de71 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:43.894903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47293f09-09a3-42a1-bdac-21da8353de71 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:43.895178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47293f09-09a3-42a1-bdac-21da8353de71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:43.907387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47293f09-09a3-42a1-bdac-21da8353de71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:43.908870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47293f09-09a3-42a1-bdac-21da8353de71 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-47293f09-09a3-42a1-bdac-21da8353de71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:43.936642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:43.937910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:43.938816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:43.938816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:43.938816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:43.938816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:43.938816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 362/724] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:43.945082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:43.988861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:44.062603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fefb1e-9b08-4b20-ac93-759af6f1911b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.062936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fefb1e-9b08-4b20-ac93-759af6f1911b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.063157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fefb1e-9b08-4b20-ac93-759af6f1911b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.064940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 363/725] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:44.070342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0a8cc-4bf4-4f80-9f45-86b27c9ac02e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.070892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0a8cc-4bf4-4f80-9f45-86b27c9ac02e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.071396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d0a8cc-4bf4-4f80-9f45-86b27c9ac02e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.100823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b1d0a8cc-4bf4-4f80-9f45-86b27c9ac02e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:44.102396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1d0a8cc-4bf4-4f80-9f45-86b27c9ac02e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:44.112443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935f452d-4891-4ff9-942e-14287f1dc841 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:44.173792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:44.174886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.175388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.175887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.176807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:44.177313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:44.178896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:44.203986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 363/726] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:44.214910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:44.219483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 364/727] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:44.256475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0a8cc-4bf4-4f80-9f45-86b27c9ac02e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:44.347330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:44.354788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:44.393376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600c9030-2bd9-406f-9738-e1cee91cf307 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:44.429291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d0a8cc-4bf4-4f80-9f45-86b27c9ac02e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:44.455511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:44.456341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.456627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.456898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.473731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:44.475191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:44.483519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:44.484367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.484651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.484930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:44.499292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:44.500327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:44.501262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:44.511615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:44.519940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:44.522044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:44.524090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:44.525602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 364/728] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:13:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:44.525745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 365/729] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:44.531170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:44.534650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:44.667331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354d0d21-e19f-46f4-8ea9-380a7552ff7c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:44.740534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbfc9b1-fef9-4e57-b824-d73de607147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:44.741490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfc9b1-fef9-4e57-b824-d73de607147a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.741778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfc9b1-fef9-4e57-b824-d73de607147a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.742058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbfc9b1-fef9-4e57-b824-d73de607147a None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.743592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfc9b1-fef9-4e57-b824-d73de607147a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:44.743945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfc9b1-fef9-4e57-b824-d73de607147a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:44.745928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbfc9b1-fef9-4e57-b824-d73de607147a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:44.752962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb2366c-62f4-430e-b130-a962b6dfb05a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:44.766680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:44.772375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 365/730] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:13:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:44.773738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbd829-5914-4de1-8d6a-1605f29e84ff None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.774025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbd829-5914-4de1-8d6a-1605f29e84ff None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.774302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffbd829-5914-4de1-8d6a-1605f29e84ff None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.776064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:44.779812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 366/731] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:44.780872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbd829-5914-4de1-8d6a-1605f29e84ff None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:44.805318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a07a180-e02f-48b7-aed1-e1e5f94b7e83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:44.806247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a180-e02f-48b7-aed1-e1e5f94b7e83 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.806540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a180-e02f-48b7-aed1-e1e5f94b7e83 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.806819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a07a180-e02f-48b7-aed1-e1e5f94b7e83 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.807574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a180-e02f-48b7-aed1-e1e5f94b7e83 None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:44.807930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a180-e02f-48b7-aed1-e1e5f94b7e83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:44.809307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a07a180-e02f-48b7-aed1-e1e5f94b7e83 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:44.818322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 366/732] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:44.819529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24c0bd-b4f6-4c59-995d-b6ccc7ec3436 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.819940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24c0bd-b4f6-4c59-995d-b6ccc7ec3436 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.820122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f24c0bd-b4f6-4c59-995d-b6ccc7ec3436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.837651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4f24c0bd-b4f6-4c59-995d-b6ccc7ec3436 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:44.838841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f24c0bd-b4f6-4c59-995d-b6ccc7ec3436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:44.888935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffbd829-5914-4de1-8d6a-1605f29e84ff None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:44.915769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 367/733] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:44.915769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecc56-36a1-4484-87cf-d201c28a4aba None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.915769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecc56-36a1-4484-87cf-d201c28a4aba None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.915769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9ecc56-36a1-4484-87cf-d201c28a4aba None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.915769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 368/734] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:44.915769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecc56-36a1-4484-87cf-d201c28a4aba None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:44.927105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf689-cdcf-41c3-bfaa-d0f9a266c7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:44.928190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf689-cdcf-41c3-bfaa-d0f9a266c7fd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.928477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf689-cdcf-41c3-bfaa-d0f9a266c7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.928758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf689-cdcf-41c3-bfaa-d0f9a266c7fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.939397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf689-cdcf-41c3-bfaa-d0f9a266c7fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:44.940883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf689-cdcf-41c3-bfaa-d0f9a266c7fd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b60cf689-cdcf-41c3-bfaa-d0f9a266c7fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:44.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:44.959980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:44.961079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:44.961079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:44.961079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:44.961079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:44.961079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 367/735] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:44.961422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea74999-f6f3-4fdc-b734-74eb6007f4f1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.961619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea74999-f6f3-4fdc-b734-74eb6007f4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.961893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea74999-f6f3-4fdc-b734-74eb6007f4f1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.967436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 368/736] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:44.967436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1640-2947-47d7-b33c-fa03c81f33bc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:44.967436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1640-2947-47d7-b33c-fa03c81f33bc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:44.967436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dc1640-2947-47d7-b33c-fa03c81f33bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:44.981914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-03dc1640-2947-47d7-b33c-fa03c81f33bc None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:44.983169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-03dc1640-2947-47d7-b33c-fa03c81f33bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:45.015099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9ecc56-36a1-4484-87cf-d201c28a4aba None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.017438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.025248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.048173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 369/737] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:45.049046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc1640-2947-47d7-b33c-fa03c81f33bc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.050289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.051237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.051672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.052131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.064248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.065803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:45.084581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:45.085479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:45.086365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:45.086365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:45.086365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:45.086365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:45.086365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 369/738] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:45.086647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.147042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dc1640-2947-47d7-b33c-fa03c81f33bc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.179264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c1effb-cfde-4235-9b58-3b5805e7fae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.179954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1effb-cfde-4235-9b58-3b5805e7fae1 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.180235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1effb-cfde-4235-9b58-3b5805e7fae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.180511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c1effb-cfde-4235-9b58-3b5805e7fae1 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.181224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1effb-cfde-4235-9b58-3b5805e7fae1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.182592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c1effb-cfde-4235-9b58-3b5805e7fae1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.184089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830541ec-6f45-495e-b647-2035693b8085 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.197832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 370/739] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:13:45 2022] POST /identity/v3/services => generated 240 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:45.199324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba231188-358b-450f-888c-6d17e1ebc0e8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.199324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba231188-358b-450f-888c-6d17e1ebc0e8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.199595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba231188-358b-450f-888c-6d17e1ebc0e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.224372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ba231188-358b-450f-888c-6d17e1ebc0e8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:45.225714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba231188-358b-450f-888c-6d17e1ebc0e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:45.233890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.234492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.234739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.235026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.236193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:45.236193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.237464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.255788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 370/740] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:45.255788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.264929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.274874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.306778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 371/741] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:45 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 07 10:13:45.307561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba231188-358b-450f-888c-6d17e1ebc0e8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.385115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98641ea8-72f8-4efc-967a-b9582f4b01e4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.417148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba231188-358b-450f-888c-6d17e1ebc0e8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.426447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.427123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.427411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.427694 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.428453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.429823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.445155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 371/742] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:13:45 2022] POST /identity/v3/services => generated 251 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:45.446284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.478970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.480550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.481592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.481878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.483469 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:45.485175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.486645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.526869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 372/743] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:45.526869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.551136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.567512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.590014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c295ae-e19d-4f69-9cf4-71a239c3a6c0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.636164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.637025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.637310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.637587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.645936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.647458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:45.665611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:45.666540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:45.667423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:45.667890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd160512-fb44-4725-95a7-2eafc9c5f7fd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.667965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 372/744] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:45.668585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.708415 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.709272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.709554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.709830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.721572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:45.729856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:45.730788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:45.731661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 373/745] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:45.732399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.789416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0972cf6c-d3de-4929-a66a-069960294156 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.824021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.830485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.831737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.832265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.832746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.841721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.844149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.849819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:45.863408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:45.864421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:13:45.865424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:45.867496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff495b7-4503-4906-ae08-96da3c800fba None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:45.872731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 373/746] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:45.873894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.912625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:45.913293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:45.913575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:45.913851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:45.914640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:45.914951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:45.917663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:45.934243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 374/747] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:45.935072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:45.991081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1c5c9f-d577-45f7-8968-edea43014afa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.030028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.031042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.031541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.032051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.033124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:46.033629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.035319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.045002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 374/748] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:46.046138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.061373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d8adb-4802-4001-bb49-7a59f05fb71c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.109081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:46.115663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.116645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.116932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.117212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.118071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:46.118608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.120723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.147827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 375/749] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jul 7 10:13:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:46.147827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.148730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:46.166762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21050931-ebcf-495d-8e3a-52b10b238f8d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.204175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.205270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.205656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.206099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/0914dada5c70437ba49688789eb31ff9/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.208409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=0914dada5c70437ba49688789eb31ff9, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.210100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.224746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 375/750] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:46 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/0914dada5c70437ba49688789eb31ff9/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:46.225723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.273377 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14f0d0-a060-4293-a35c-e0c01100ba87 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.321677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.322879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.323347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.323822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.344577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee39fd0-4601-4605-abd0-c30ccbacb450 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.346894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.348686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.361501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:46.383675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:46.384841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:46.385963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:46.385963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:46.385963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:46.385963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:46.391782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.391782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.391782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.391782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.393899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 376/751] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:46.399305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.475772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.475772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.495938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:46.582686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525f0db7-cd1c-42d7-b5eb-cb15515fe250 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.587483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 376/752] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:13:46.593386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.609023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:46.622630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.623484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.623900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.624278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.625109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:46.625518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.627001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.640598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 377/753] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:46.645482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.702937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce696517-fea9-40f0-ba04-85229a286643 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.742788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.743904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.744376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.744806 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.756056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.757179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a23854-23b3-4b29-a215-965c61a757c7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.757999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.772379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:46.778757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:46.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:46.781483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:46.781483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:46.781483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:46.781483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:46.783474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 377/754] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:46.789104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.797591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.798535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.798908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.799305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.808609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.810201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:46.825313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:46.827070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:46.828290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:46.830057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 378/755] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:46.841054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:46.867036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:46.907329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790029e4-5c25-4298-85b1-787fd26cb7f9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:46.945745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:46.946747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:46.947180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:46.947646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:46.948589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:46.949039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:46.950800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:46.964444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 378/756] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:46.970573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:47.016798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc6265-8f10-49ee-beed-089a297ae8f0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:47.077908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:47.078809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.079104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.079476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.080449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:47.080860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:47.082374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:47.082511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0526e42-40bd-44e0-b731-24de0c39baf8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:47.110620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:47.113628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 379/757] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:47.118616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:47.121938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:47.123067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.123488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.123936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.131795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:47.133474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:47.149519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:47.150596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:47.151713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:47.153986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 379/758] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:47.160665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:47.176937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:47.275575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c437af67-17f3-43f2-a3a2-544abc187936 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:47.295048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982453e-bc57-4a35-bf7f-cfa572b1917b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:47.314800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:47.315918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.316417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.316957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.318094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:47.318614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:47.320450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:47.334084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 380/759] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:47.340147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:47.353039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:47.353702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.353984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.354256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.393188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:47.453735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:47.455204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:47.455648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f232e-4e0e-4d4e-a9e1-ba4a8747c08b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:47.485714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:47.495683 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6d0039-1a6e-4fc7-a10b-0236e64f3405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:47.496767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d0039-1a6e-4fc7-a10b-0236e64f3405 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.497287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d0039-1a6e-4fc7-a10b-0236e64f3405 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.497857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6d0039-1a6e-4fc7-a10b-0236e64f3405 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.499420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d0039-1a6e-4fc7-a10b-0236e64f3405 None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=e74571f351a44ef48c8f51695a4e42d8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:47.499991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d0039-1a6e-4fc7-a10b-0236e64f3405 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:47.507260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6d0039-1a6e-4fc7-a10b-0236e64f3405 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:47.518729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 381/760] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:47 2022] GET /identity/v3/role_assignments?user.id=e74571f351a44ef48c8f51695a4e42d8&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:47.525603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdd20b-e4fc-4ebc-92ef-1bbce73a9623 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.526168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdd20b-e4fc-4ebc-92ef-1bbce73a9623 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.526658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cdd20b-e4fc-4ebc-92ef-1bbce73a9623 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.532920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 382/761] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:47.539557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031c3d0-9eb2-430a-b66a-0a374947e471 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.540214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031c3d0-9eb2-430a-b66a-0a374947e471 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.540822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8031c3d0-9eb2-430a-b66a-0a374947e471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.567650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 380/762] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 10:13:47.571192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8031c3d0-9eb2-430a-b66a-0a374947e471 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:47.572853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:47.582611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8031c3d0-9eb2-430a-b66a-0a374947e471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:47.697124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 383/763] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:13:47.731114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2227e-77a2-4642-bdb9-c80f887518ed None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.731408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2227e-77a2-4642-bdb9-c80f887518ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.731692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b2227e-77a2-4642-bdb9-c80f887518ed None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.733766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 384/764] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:13:47.734319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2227e-77a2-4642-bdb9-c80f887518ed None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:47.754405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa55dde-1943-4002-817b-e64e99cba630 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:47.789721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:47.790648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:47.798563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9e5cf-a744-421f-8a8f-bd59138cff2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:47.799685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9e5cf-a744-421f-8a8f-bd59138cff2f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.799999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9e5cf-a744-421f-8a8f-bd59138cff2f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.800279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9e5cf-a744-421f-8a8f-bd59138cff2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.814565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9e5cf-a744-421f-8a8f-bd59138cff2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:47.816032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9e5cf-a744-421f-8a8f-bd59138cff2f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-09c9e5cf-a744-421f-8a8f-bd59138cff2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:47.835863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:47.836815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:47.837710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:47.837710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:47.837710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:47.837710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:47.837710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 381/765] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:47.839309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064442d-d56d-4290-9473-f72ec6b6d8ab None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.839309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064442d-d56d-4290-9473-f72ec6b6d8ab None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.839309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4064442d-d56d-4290-9473-f72ec6b6d8ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.858240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4064442d-d56d-4290-9473-f72ec6b6d8ab None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:47.859438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4064442d-d56d-4290-9473-f72ec6b6d8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:47.868982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b2227e-77a2-4642-bdb9-c80f887518ed None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:47.916329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:47.917509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:47.918007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:47.918573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] PATH_INFO: `/v3/services/network` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:47.920056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:47.922181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:47.933915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 382/766] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:47.934760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064442d-d56d-4290-9473-f72ec6b6d8ab None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:47.939845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:47.940801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:47.941641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:47.945227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 385/767] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:47 2022] GET /identity/v3/services/network => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:47.952238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.059161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4064442d-d56d-4290-9473-f72ec6b6d8ab None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.075121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.077292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae58aa-9115-4383-9673-2c697d4c50f5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.081374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.099757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.100455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.100737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.101016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.101783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.102133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.103611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.116512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 383/768] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.117471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.124889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.125845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.126340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.127370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.128337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] RBAC: Adding query filter params (name=network) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.128857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.131375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.138558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 386/769] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:13:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.140126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.230340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c697c3-aa4f-4a6e-b806-a9be8d0d17e5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.267536 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.268342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.268625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.268900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.269672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.271244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.278601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82e079d-9259-423a-a792-4b32fe008f9c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:48.283969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:48.284899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:48.285735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 384/770] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:48.286577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.328330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.329821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.330809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.331302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.332136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.341434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.343460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:48.363567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:48.364511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:48.365424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:48.369225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 387/771] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:48.370604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.402405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.404553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2edab56-73de-4b9c-bc54-52f5abc70d4c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.449044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.449697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.449981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.450255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.450953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.451297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.452703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.458097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 385/772] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.458893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.500180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a0124-9d8d-4ae2-89db-2d5e73922de9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.545907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.546675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.547033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.547389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.548226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.548605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.550161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.555720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 388/773] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.556723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.575924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ab4375-cc01-4695-89f0-f9d01216c569 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.577801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.613598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.614560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.615047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.615515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.616517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.617058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.618665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.631964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 386/774] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.633116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.661817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.675713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7e601-8824-419b-a89e-e053fee593ac None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.715423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.716152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.716435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.717256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.718124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] RBAC: Adding query filter params (interface=public, service_id=317ecea121754cfaad5fb0cb84224718, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.718480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.720468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.728557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 389/775] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/endpoints?service_id=317ecea121754cfaad5fb0cb84224718&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.729588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.756181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7368f59-05cf-4382-9b60-10e565ce49b6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.796797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.797612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.798105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.798501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.799452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.799956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.801436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.807261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 387/776] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.808224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.837535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.858280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a059fa-8c57-4c83-8eb7-72c65efecff2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.899815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.900954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.901414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.901995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.903262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=3e2e080fd7144086ac2cf5628a7a68ed) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:48.903729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.905918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.914097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 390/777] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/role_assignments?user.id=3e2e080fd7144086ac2cf5628a7a68ed&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.915148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:48.921016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b28bb-c2d1-44fa-b879-c36b00cc3fc2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:48.944154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:48.967404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c272800d-ed27-406d-b867-9b697df20a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:48.968507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272800d-ed27-406d-b867-9b697df20a81 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.969016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272800d-ed27-406d-b867-9b697df20a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.969513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c272800d-ed27-406d-b867-9b697df20a81 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.970408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272800d-ed27-406d-b867-9b697df20a81 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:48.972257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c272800d-ed27-406d-b867-9b697df20a81 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:48.979515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 388/778] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/services => generated 772 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:48.981739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0354f9a-a5bb-4b02-b8f1-e802dea6d3f4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.982272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0354f9a-a5bb-4b02-b8f1-e802dea6d3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.983366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0354f9a-a5bb-4b02-b8f1-e802dea6d3f4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:48.985671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 389/779] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:48.991836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06be3d-0a44-4f31-848e-af419ac2602a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:48.991836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06be3d-0a44-4f31-848e-af419ac2602a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:48.992136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06be3d-0a44-4f31-848e-af419ac2602a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.032858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-af06be3d-0a44-4f31-848e-af419ac2602a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:49.034023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af06be3d-0a44-4f31-848e-af419ac2602a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:49.045354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096593f1-c472-4a37-8cd1-81ab60cee343 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:49.098281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:49.098966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.099252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.099531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.100390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] RBAC: Adding query filter params (interface=public, service_id=8d5b50cbc7ca462ab2e1c7627f5c2fe7, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:49.100706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:49.102125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:49.110940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 391/780] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:13:48 2022] GET /identity/v3/endpoints?service_id=8d5b50cbc7ca462ab2e1c7627f5c2fe7&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:49.116092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:49.166305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 390/781] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:49.206390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06be3d-0a44-4f31-848e-af419ac2602a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:49.211695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:49.237544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85285006-576f-4396-8ce1-6a73d4e6cc5c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:49.239261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:49.278434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:49.279167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.279990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.280313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.280962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:49.282476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:49.291979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 392/782] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:13:49 2022] GET /identity/v3/services => generated 772 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:49.345449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06be3d-0a44-4f31-848e-af419ac2602a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:49.385229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:49.386294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.386724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.387144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.399878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:49.408844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:49.410044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:49.411133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:49.413815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 391/783] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:49.419521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:49.549587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca72cdc7-cf8f-4230-be4a-e3313c89ed3b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:49.585338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfac1e6-b0cd-4c94-b390-03c2d4e2b090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:49.586022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac1e6-b0cd-4c94-b390-03c2d4e2b090 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.586312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac1e6-b0cd-4c94-b390-03c2d4e2b090 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.586595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfac1e6-b0cd-4c94-b390-03c2d4e2b090 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.587350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfac1e6-b0cd-4c94-b390-03c2d4e2b090 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:49.587663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfac1e6-b0cd-4c94-b390-03c2d4e2b090 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:49.589060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfac1e6-b0cd-4c94-b390-03c2d4e2b090 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:49.602372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 393/784] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:49.603756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:49.693128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:49.713320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908c668-7396-4f43-9a59-4af1c637caca None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:49.753216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:49.754021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.754307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.754587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.775466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:49.776956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:49.792105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20a8e1-c716-483d-9699-fdd1779d8f8d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.792458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20a8e1-c716-483d-9699-fdd1779d8f8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.792746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d20a8e1-c716-483d-9699-fdd1779d8f8d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.794119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 394/785] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:13:49.799753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47714e-61d8-43d9-a39e-38f291550ac2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.800010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47714e-61d8-43d9-a39e-38f291550ac2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.800296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47714e-61d8-43d9-a39e-38f291550ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:49.800798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:49.801651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:49.802539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:49.802539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:49.802539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:49.802539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:49.802539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 392/786] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:49.807664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:49.821478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7b47714e-61d8-43d9-a39e-38f291550ac2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:49.822685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7b47714e-61d8-43d9-a39e-38f291550ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:49.825657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:49.895461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 395/787] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:13:49.938865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36eff6-b979-459b-85fe-8ba6866d4af1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:49.950191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47714e-61d8-43d9-a39e-38f291550ac2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:49.977102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:49.987335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:49.988261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:49.988547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:49.988827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:49.990695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:49.991009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:49.992414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:50.019399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 393/788] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:50.022127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:50.095006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47714e-61d8-43d9-a39e-38f291550ac2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:50.132633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:50.133439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:50.133723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:50.134373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:50.145404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:50.156065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:50.161944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:50.162886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:50.163708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:50.172976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 396/789] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:50.175854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:50.277334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bed1a9-3c7e-456f-86fb-7dba3008cefe None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:50.347308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55292578-c6c1-4fde-a4f3-ec523aba6e5f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:50.350476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:50.351880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:50.352541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:50.353651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:50.368762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:50.370649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:50.391943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:50.392978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:50.399170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 394/790] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:13:50 2022] GET /identity/v3/users/nova => generated 82 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:50.402343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:50.416068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:50.417061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:50.417558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:50.418046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:50.419005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:50.419532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:50.422165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:50.441910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 397/791] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:50.447785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:50.513213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:50.570918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:50.606268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66beb14f-3982-4481-9f2e-a39ec5472e13 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:50.648334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602058c8-42a0-4f7e-a8c5-fd4b0563d8ca None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:50.676765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:50.677447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:50.677734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:50.678015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:50.678857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:50.679170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:50.680638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:50.717367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 395/792] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:13:50 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:50.719093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:50.719961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:50.720277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:50.720531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:50.723497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:50.745302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:50.746787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:50.776421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:50.777405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:50.778284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:50.778284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:50.778284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:50.778284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:50.778284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 398/793] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:50.789498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:50.866899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9268f9c-5464-4329-8403-a6209c1fef50 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:50.878163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:50.905732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:50.906670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:50.907107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:50.907535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:50.948861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacac641-b315-4b94-b821-189ed2a0ba4b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:50.964415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:50.989738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:50.991921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.024498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.025312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.025711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.026120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.026954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:51.027524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:51.028903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.051410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 399/794] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:51.057012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.109188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 396/795] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:13:51.115714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.152642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:51.217358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4dc4b4-d382-422a-b64a-aab227150041 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:51.257798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.259018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.259537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.260049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.270165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:51.273428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:51.290825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:51.291816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:13:51.292766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:51.297435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 400/796] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:51.302767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.358803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:51.394164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529fd2f5-07d4-43c0-8433-6c7cb0500b96 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:51.430558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7881e1-77a9-4fc4-88da-a1995ece0477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.431602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7881e1-77a9-4fc4-88da-a1995ece0477 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.431602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7881e1-77a9-4fc4-88da-a1995ece0477 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.431827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7881e1-77a9-4fc4-88da-a1995ece0477 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.445187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7881e1-77a9-4fc4-88da-a1995ece0477 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:51.446647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7881e1-77a9-4fc4-88da-a1995ece0477 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.452517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b582406-4932-4173-86e4-a8cfe9b9b81e None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-dc7881e1-77a9-4fc4-88da-a1995ece0477 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:51.469102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:51.470071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:51.470971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:51.470971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:51.470971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:51.470971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:51.474925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 397/797] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:51.476279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8f51-14d0-4ad0-b1c1-ec6ecdd76de0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.476567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8f51-14d0-4ad0-b1c1-ec6ecdd76de0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.476847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9d8f51-14d0-4ad0-b1c1-ec6ecdd76de0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.478355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 398/798] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:51.479059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d8f51-14d0-4ad0-b1c1-ec6ecdd76de0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.503518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956ac17-a804-4d9d-8496-cb1e72b42217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.504612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ac17-a804-4d9d-8496-cb1e72b42217 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.505114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ac17-a804-4d9d-8496-cb1e72b42217 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.505603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ac17-a804-4d9d-8496-cb1e72b42217 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.507420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956ac17-a804-4d9d-8496-cb1e72b42217 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:51.508042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956ac17-a804-4d9d-8496-cb1e72b42217 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:51.509885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956ac17-a804-4d9d-8496-cb1e72b42217 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.530154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 401/799] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:51.536119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8399221-6d95-4ac5-a180-22c568568527 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.536119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8399221-6d95-4ac5-a180-22c568568527 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.536119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8399221-6d95-4ac5-a180-22c568568527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.550413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b8399221-6d95-4ac5-a180-22c568568527 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:51.551588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8399221-6d95-4ac5-a180-22c568568527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:51.556352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:51.598281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9d8f51-14d0-4ad0-b1c1-ec6ecdd76de0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:51.625240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 402/800] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:51.626681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5875c47-1376-4a3c-a8f2-4a2f186a64f0 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.626980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5875c47-1376-4a3c-a8f2-4a2f186a64f0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.627260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5875c47-1376-4a3c-a8f2-4a2f186a64f0 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.631796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 403/801] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:51.631796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5875c47-1376-4a3c-a8f2-4a2f186a64f0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.645200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0430b497-a547-48d9-8ed3-c62b9e367a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.645987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b497-a547-48d9-8ed3-c62b9e367a5c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.646372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b497-a547-48d9-8ed3-c62b9e367a5c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.646750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0430b497-a547-48d9-8ed3-c62b9e367a5c None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.647574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430b497-a547-48d9-8ed3-c62b9e367a5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:51.648011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430b497-a547-48d9-8ed3-c62b9e367a5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:51.649713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0430b497-a547-48d9-8ed3-c62b9e367a5c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.660138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:51.662543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 399/802] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:51.663807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da0296-1294-497e-a748-1ce0772901a1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.664097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da0296-1294-497e-a748-1ce0772901a1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.664390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5da0296-1294-497e-a748-1ce0772901a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.681345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e5da0296-1294-497e-a748-1ce0772901a1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:51.682481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e5da0296-1294-497e-a748-1ce0772901a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:51.740366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5875c47-1376-4a3c-a8f2-4a2f186a64f0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:51.754200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 400/803] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:51.756755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da0296-1294-497e-a748-1ce0772901a1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.782113 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.782908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.783192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.783471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.793835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:51.795317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:51.813477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:51.814499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:51.815444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:51.815444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:51.815444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:51.815444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:51.815663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:51.815735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 404/804] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:51.815842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.865775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5da0296-1294-497e-a748-1ce0772901a1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:51.889808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:51.899711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ede2914-4653-4f79-b5fa-97d181cfeb1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.900520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2914-4653-4f79-b5fa-97d181cfeb1f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.900803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2914-4653-4f79-b5fa-97d181cfeb1f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.901079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ede2914-4653-4f79-b5fa-97d181cfeb1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.911179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ede2914-4653-4f79-b5fa-97d181cfeb1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:51.914041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf1c17-bb44-42ca-bcf5-a7e436775c87 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4ede2914-4653-4f79-b5fa-97d181cfeb1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:51.918919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:51.919941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:51.920789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:51.923886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 401/805] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:51.926431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f776de-f98d-49c7-8eba-b6b644313b44 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.926716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f776de-f98d-49c7-8eba-b6b644313b44 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.926994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f776de-f98d-49c7-8eba-b6b644313b44 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.928300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 402/806] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:51.928984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f776de-f98d-49c7-8eba-b6b644313b44 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:51.949752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:51.950487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:51.950855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:51.951628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:51.958298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:51.959770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:51.973384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:51.974372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 405/807] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:51.975508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.035078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f776de-f98d-49c7-8eba-b6b644313b44 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.047843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.069048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.069829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.070108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.070380 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] PATH_INFO: `/v3/services/network` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.071027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.072413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.075366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ea0cb8-26bf-48d1-ad0d-40df522fe744 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:52.084041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:52.085013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 07 10:13:52.085846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:52.088054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 403/808] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:52.088874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.109031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5454870-f3cb-477b-917e-f52169708574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.109695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5454870-f3cb-477b-917e-f52169708574 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.109980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5454870-f3cb-477b-917e-f52169708574 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.110257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5454870-f3cb-477b-917e-f52169708574 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.110979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5454870-f3cb-477b-917e-f52169708574 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:52.111287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5454870-f3cb-477b-917e-f52169708574 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.112977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5454870-f3cb-477b-917e-f52169708574 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.114398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.126069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 406/809] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:52.127271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8336f0-da1f-4a15-8482-01396f313503 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.127555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8336f0-da1f-4a15-8482-01396f313503 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.127856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8336f0-da1f-4a15-8482-01396f313503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.143430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6b8336f0-da1f-4a15-8482-01396f313503 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:52.144661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b8336f0-da1f-4a15-8482-01396f313503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:52.190134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a0543-8b60-458b-b75e-1bb7b369aa8e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.211301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 407/810] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:52.212069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8336f0-da1f-4a15-8482-01396f313503 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.230388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.231116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.231705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.232471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.233837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:52.234202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.235702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.244182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 404/811] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:52.244804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.283772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.309198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8336f0-da1f-4a15-8482-01396f313503 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.333250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.343212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9545d2e-c0aa-4f40-b3a2-a9413b74c775 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.344414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.345081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.345370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.345652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.346441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:52.346755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.348260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.360083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 408/812] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:52.360868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.378204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.378871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.379155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.379436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.380166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] RBAC: Adding query filter params (name=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:52.380474 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.381841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.386328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 405/813] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:52.387091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.462539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ddb205-6954-4a30-8dde-c3039e367c97 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.486498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87352c8-76a2-45fa-8ad1-66cdb552ee94 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.487994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.496132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.496929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.497213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.497490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.504109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.505727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:52.519891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:52.520886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:52.521761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:52.522179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.522277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 409/814] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:52.522349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.522349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.522349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.522533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.522614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.523919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:52.538572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:52.539701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 07 10:13:52.540757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:52.543881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 406/815] 10.222.0.16 () {56 vars in 1129 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:52.545693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.571208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.625495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da70678a-a3a0-4ab8-9b88-4549f1af5cfd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.642232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4fb0e-8144-483e-a895-2fff6d9690ee None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.659460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.660281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.660569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.660848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.671807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.673321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.675559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.676246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.676556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.676836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.677656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=0914dada5c70437ba49688789eb31ff9) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:52.677970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.679448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.685383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 407/816] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/role_assignments?user.id=0914dada5c70437ba49688789eb31ff9&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:52.686157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:52.691886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:52.693005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:52.694125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:52.694125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:52.694125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:52.694125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:52.694754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 410/817] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:52.695611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.731546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.809687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.813452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5f906-8b05-493e-b3e5-6401d09e1050 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.817254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a4b7d-82c1-4d9c-9fae-c58560b7affb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.853305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.853972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.854636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.854636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.855692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] RBAC: Adding query filter params (type=network) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:52.855692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.857945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.858056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:52.858682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:52.858970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:52.859244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:52.860067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:52.860379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:52.861804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:52.862922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 408/818] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:52.863538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.875105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 411/819] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:52.875813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:52.963898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:52.971257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09665c90-d0e0-42ef-b18e-953275492016 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:52.988030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8134e4b-e667-418e-a241-fd5629d9048a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.006488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.007158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.007444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.007725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.008463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:53.008768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.010147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.015180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 409/820] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:53.015960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:53.032690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.033848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.034144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.034564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.035424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:53.036003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.037767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.054569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 412/821] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:53.055669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:53.075555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:53.125530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b846263-5748-4b1b-844b-8a5f2ad2b180 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.161410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.162072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.162357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.162634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.163054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b13e9e-7091-42c4-aa24-0749be14a5e9 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.163378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.164800 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.190687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 410/822] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:13:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:53.191539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:53.221031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.222533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.223045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.223732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/e74571f351a44ef48c8f51695a4e42d8/roles/54eab69c526d4d53807c7d0f9b1862c7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.243644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=e74571f351a44ef48c8f51695a4e42d8, role_id=54eab69c526d4d53807c7d0f9b1862c7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.245715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.264526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 413/823] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:53 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/e74571f351a44ef48c8f51695a4e42d8/roles/54eab69c526d4d53807c7d0f9b1862c7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:53.266054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:53.337809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:53.358573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:53.421860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bd9518-7237-40b3-8813-54275fdf8ff8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.449929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4525a97d-cf61-4539-8012-dde52cf60682 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.469339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.470556 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.470912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.471260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.482529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.484898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.491859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.492542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.492830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.493108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.493876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:53.494133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.495528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.502244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 411/824] 10.222.0.16 () {56 vars in 1124 bytes} [Thu Jul 7 10:13:53 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:53.503695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:53.504762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:53.505698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 414/825] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:53.510191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:53.512851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:53.614927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:53.638065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082af1e9-3aee-41e6-9c81-5b70f8019974 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.655516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb26ffc-8a4a-4f73-9509-38bce57be211 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.673980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.674768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.675155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.675532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.676411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.677882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.684034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:53.699157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.699529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 412/826] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:13:53 2022] POST /identity/v3/endpoints => generated 346 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:13:53.699871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.700155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.700525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.701455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:53.701769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.703326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.725006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 415/827] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jul 7 10:13:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:13:53.730772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:53.856449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9a240f-12c8-4dc5-8e57-3a5b3a4e6f97 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:53.886959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:53.893679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:53.894613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:53.895115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:53.895586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:53.907801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:53.909415 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:53.927852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:53.928892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:53.929783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:53.929783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:53.929783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:53.929783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:53.930008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 413/828] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:53.935975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:54.051055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248dda7-a010-477f-ac92-8e32d63ea569 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:54.088445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:54.089522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:54.090028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:54.090511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:54.092040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:54.092557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:54.094272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:54.106919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 416/829] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:54.112375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:54.217564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6788d8ef-ba9f-4007-b99d-9619085f1f47 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:54.260093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:54.260882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:54.261163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:54.261434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:54.268678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:54.270254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:54.285166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:54.286135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:54.287017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 414/830] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:54.292589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:54.337396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:54.399555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433bca1-a648-432d-afbc-66f9ce8929f3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:54.434268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304c9d9d-b4c9-4cbc-8b76-00a3e0874183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:54.435846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c9d9d-b4c9-4cbc-8b76-00a3e0874183 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:54.436364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c9d9d-b4c9-4cbc-8b76-00a3e0874183 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:54.437095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c9d9d-b4c9-4cbc-8b76-00a3e0874183 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:54.438085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c9d9d-b4c9-4cbc-8b76-00a3e0874183 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:54.438601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c9d9d-b4c9-4cbc-8b76-00a3e0874183 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:54.440342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304c9d9d-b4c9-4cbc-8b76-00a3e0874183 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:54.568528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 417/831] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:54.574739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:54.625637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:54.685718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e37c8d-e54e-46e3-b2c8-3624d94f956f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:54.729115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6800279b-4210-4273-b2e5-0e471e34810a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:54.730011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800279b-4210-4273-b2e5-0e471e34810a None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:54.730298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800279b-4210-4273-b2e5-0e471e34810a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:54.730578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6800279b-4210-4273-b2e5-0e471e34810a None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/3e2e080fd7144086ac2cf5628a7a68ed/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:54.732618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800279b-4210-4273-b2e5-0e471e34810a None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=3e2e080fd7144086ac2cf5628a7a68ed, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:54.734057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6800279b-4210-4273-b2e5-0e471e34810a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:54.749799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 415/832] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:13:54 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/3e2e080fd7144086ac2cf5628a7a68ed/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:13:54.901372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29117f2-39e7-4d72-b37a-7b6e7066bf89 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:54.901690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29117f2-39e7-4d72-b37a-7b6e7066bf89 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:54.901935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29117f2-39e7-4d72-b37a-7b6e7066bf89 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:54.903443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 418/833] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:54.908806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb12ca4-421f-4c60-9156-9a100ccfe13d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:54.909099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb12ca4-421f-4c60-9156-9a100ccfe13d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:54.909379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb12ca4-421f-4c60-9156-9a100ccfe13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:54.928953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fdb12ca4-421f-4c60-9156-9a100ccfe13d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:54.930162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fdb12ca4-421f-4c60-9156-9a100ccfe13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:55.001388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 416/834] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:55.022635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29117f2-39e7-4d72-b37a-7b6e7066bf89 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:55.128129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:55.132214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29117f2-39e7-4d72-b37a-7b6e7066bf89 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:55.167633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:55.168488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.168775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.169106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.170363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:55.171730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:55.184422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:55.185398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:55.186229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 419/835] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jul 7 10:13:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:55.191695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb12ca4-421f-4c60-9156-9a100ccfe13d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:55.226381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:55.300564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb12ca4-421f-4c60-9156-9a100ccfe13d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:55.335839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6034c-22bc-4124-b73e-fe7be1b00d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:55.336879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6034c-22bc-4124-b73e-fe7be1b00d62 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.337369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6034c-22bc-4124-b73e-fe7be1b00d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.337868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6034c-22bc-4124-b73e-fe7be1b00d62 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.338767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6034c-22bc-4124-b73e-fe7be1b00d62 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:55.339287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6034c-22bc-4124-b73e-fe7be1b00d62 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:55.340890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6034c-22bc-4124-b73e-fe7be1b00d62 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:55.346593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 417/836] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:13:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:55.352458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:55.456792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f172bd1d-1380-4eab-9a4c-a5104c2768bb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:55.492367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d1c1be-7faa-48b6-b528-6a9116ed8c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:55.493453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c1be-7faa-48b6-b528-6a9116ed8c88 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.493968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c1be-7faa-48b6-b528-6a9116ed8c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.494450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d1c1be-7faa-48b6-b528-6a9116ed8c88 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.495357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1c1be-7faa-48b6-b528-6a9116ed8c88 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:55.495903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1c1be-7faa-48b6-b528-6a9116ed8c88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:55.497516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d1c1be-7faa-48b6-b528-6a9116ed8c88 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:55.502774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 420/837] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:13:55 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:55.528148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69def558-be0c-41c1-8175-b224c678e3b9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.528488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69def558-be0c-41c1-8175-b224c678e3b9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.529275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69def558-be0c-41c1-8175-b224c678e3b9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.530982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 418/838] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:55.536740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9db0e-1f2a-48e4-b087-050623011a04 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.537032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9db0e-1f2a-48e4-b087-050623011a04 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.537294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9db0e-1f2a-48e4-b087-050623011a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.559082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d3f9db0e-1f2a-48e4-b087-050623011a04 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:55.560813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3f9db0e-1f2a-48e4-b087-050623011a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:55.592399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:55.636795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 421/839] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:55.661598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69def558-be0c-41c1-8175-b224c678e3b9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:55.761186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:55.768807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69def558-be0c-41c1-8175-b224c678e3b9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:55.807480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fe5b8b-061e-42c9-9e60-6a81d5a693ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:55.808384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe5b8b-061e-42c9-9e60-6a81d5a693ed None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.808676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe5b8b-061e-42c9-9e60-6a81d5a693ed None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.808958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fe5b8b-061e-42c9-9e60-6a81d5a693ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.820266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fe5b8b-061e-42c9-9e60-6a81d5a693ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a3fe5b8b-061e-42c9-9e60-6a81d5a693ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:55.828761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:55.829696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:55.830550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 419/840] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:13:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:55.837565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9db0e-1f2a-48e4-b087-050623011a04 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:55.881721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecbbb9-aaa6-4400-8245-a0be5065dade None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.882032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecbbb9-aaa6-4400-8245-a0be5065dade None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.882277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecbbb9-aaa6-4400-8245-a0be5065dade None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.887580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 420/841] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:55.889870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1142-a254-49c8-b6b2-dd2abd7bdc97 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.890165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1142-a254-49c8-b6b2-dd2abd7bdc97 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.890445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938c1142-a254-49c8-b6b2-dd2abd7bdc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.908626 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-938c1142-a254-49c8-b6b2-dd2abd7bdc97 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:55.909834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-938c1142-a254-49c8-b6b2-dd2abd7bdc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:55.955632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9db0e-1f2a-48e4-b087-050623011a04 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:55.987869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 421/842] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:13:55.996938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:55.998150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:55.998435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:55.998717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:55.999474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:55.999807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.001180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.004271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1142-a254-49c8-b6b2-dd2abd7bdc97 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.015346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 422/843] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:13:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:56.021279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.053293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:56.104903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:56.127438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938c1142-a254-49c8-b6b2-dd2abd7bdc97 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.142971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b6f9b-1433-428d-ae0a-4dd581d4b14d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.172097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.173716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.174159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.174616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.175420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.176967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.177307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.178107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.178393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.178671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.191156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:56.192532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:56.193729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:13:56.194778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:56.195701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.197850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 422/844] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:56.205041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:56.217624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:56.218573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:56.219465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:56.219465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:56.219465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:56.219465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:56.219465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 423/845] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:56.226298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.319279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:56.340238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3a3137-7749-4562-b86a-cee28583a528 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.357166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05299ce-1141-4234-8375-8bb1f954a597 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.385826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.386561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.386943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.387490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.388111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:56.388504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.389932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.395891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 423/846] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:56.400476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.403302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.404064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.404342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.404614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.405698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:56.405698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.407090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.420455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 424/847] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:56.421057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:56.425769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.516009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e352cef-ed10-4757-814c-27ed991fbb96 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.553136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.553848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.554131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.554297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0db6a5-b182-4ff2-a8fb-f732ed6c916f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.554395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.555080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:56.555382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.556992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.562566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 424/848] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:56.589405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:56.597245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.598084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.598768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.599524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.608081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.609479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:56.626780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:56.627810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:56.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 425/849] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:56.633828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.724441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:56.745692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f95ba9-cbb4-41ed-a980-f4cffb87624a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.781269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2915d3-ca90-47d8-8cda-155420618772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.781943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2915d3-ca90-47d8-8cda-155420618772 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.782221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2915d3-ca90-47d8-8cda-155420618772 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.782490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2915d3-ca90-47d8-8cda-155420618772 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.783314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2915d3-ca90-47d8-8cda-155420618772 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:56.783622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2915d3-ca90-47d8-8cda-155420618772 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:56.785067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2915d3-ca90-47d8-8cda-155420618772 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:56.803218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 425/850] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:56.808697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.842707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618434-3261-4db4-867b-86f99dcfc0bf None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.843028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618434-3261-4db4-867b-86f99dcfc0bf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.843251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f618434-3261-4db4-867b-86f99dcfc0bf None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.844753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 426/851] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:13:56.850370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595058c-8cbb-4daf-b921-6f8093e7cfb1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.850852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595058c-8cbb-4daf-b921-6f8093e7cfb1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.851231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c595058c-8cbb-4daf-b921-6f8093e7cfb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:56.871183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c595058c-8cbb-4daf-b921-6f8093e7cfb1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:56.872884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c595058c-8cbb-4daf-b921-6f8093e7cfb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:56.933241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194d028b-3a76-4a31-8ec0-1716ff2eba3d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:56.949664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 427/852] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:56.968747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595058c-8cbb-4daf-b921-6f8093e7cfb1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:56.974102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:56.974951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:56.975378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:56.975832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.046690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.053633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.065581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.067130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.093499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c595058c-8cbb-4daf-b921-6f8093e7cfb1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.138651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.139679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.140352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.140781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.153839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:13:57.164262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:57.165396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:13:57.166424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:57.169721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 428/853] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:57.175333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.188172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 426/854] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:13:57.188976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.287811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.304179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1969ed24-12ab-4df3-8e7d-4baeee0eebe8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.308401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b696ad85-4fde-4913-b3be-4c80b226ad73 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.337295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.347507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.348027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.348310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.348587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.349328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:57.349635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.351729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.353995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.355469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.355983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.356470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.365920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 429/855] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:57.366950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.375790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.377492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:57.402344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:57.403207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:57.404636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:57.404636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:57.404636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:57.404636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:57.410114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 427/856] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:57.411125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.494267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e0cb62-e6b6-42d0-ac52-3243befdfb11 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.514441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b23a856-b2f0-49a4-af5e-2c51a5b7ea76 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.540910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.541704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.541982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.542251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.553843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.555292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.559448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.560450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.560929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.561699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.563175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:57.563717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.565992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:57.573506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:57.574433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:57.575326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:57.575326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:57.575326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:57.575326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:57.578882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 430/857] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:57.580484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.581654 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 428/858] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:57.588344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.593513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.639590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.697253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b00a30-07c1-4569-9000-fd3223bc941d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.713510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b58abb8-84f5-4ce7-8021-c3f6e228ed7d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.735060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84904b0b-dcaa-4ba8-976a-c923e8d40529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.735876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904b0b-dcaa-4ba8-976a-c923e8d40529 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.736166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904b0b-dcaa-4ba8-976a-c923e8d40529 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.736440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904b0b-dcaa-4ba8-976a-c923e8d40529 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.743225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84904b0b-dcaa-4ba8-976a-c923e8d40529 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.744667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84904b0b-dcaa-4ba8-976a-c923e8d40529 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.750861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.751540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.751840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.752122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.752841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:57.753153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.754514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-84904b0b-dcaa-4ba8-976a-c923e8d40529 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:57.758812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:57.759652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:57.760851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:57.764536 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 429/859] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:13:57.766346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a9d2-478c-4ee9-93ed-f4609bd03b22 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.766638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a9d2-478c-4ee9-93ed-f4609bd03b22 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.766914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d1a9d2-478c-4ee9-93ed-f4609bd03b22 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.767487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 431/860] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:57.772117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.772241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 430/861] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:57.776652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1a9d2-478c-4ee9-93ed-f4609bd03b22 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:57.851139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.873964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:57.877926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b90a9-0058-4b28-90c6-aaa94fd9c204 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.892051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d1a9d2-478c-4ee9-93ed-f4609bd03b22 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:57.915229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b349b83-c878-48ce-89ad-2e7ca7b92afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.915915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b349b83-c878-48ce-89ad-2e7ca7b92afc None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.916243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b349b83-c878-48ce-89ad-2e7ca7b92afc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.916464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b349b83-c878-48ce-89ad-2e7ca7b92afc None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.917230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b349b83-c878-48ce-89ad-2e7ca7b92afc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:57.917539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b349b83-c878-48ce-89ad-2e7ca7b92afc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.919107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b349b83-c878-48ce-89ad-2e7ca7b92afc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.927713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:57.928534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.928821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.929105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.931410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 432/862] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:57.932793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8f756-a906-4d1e-8f49-290621086ad4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:57.933077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8f756-a906-4d1e-8f49-290621086ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:57.933351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee8f756-a906-4d1e-8f49-290621086ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:57.941594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:57.943034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:57.952530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fee8f756-a906-4d1e-8f49-290621086ad4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:57.953649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fee8f756-a906-4d1e-8f49-290621086ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:13:57.957977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:57.958882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:57.960239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 431/863] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:57.960835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:58.029003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 433/864] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:58.029685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8f756-a906-4d1e-8f49-290621086ad4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:58.074707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20442b4e-3f14-4d9a-8738-f8c2bec85e54 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:58.102354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:58.114230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:58.115037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.115425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.115839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.116775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=e74571f351a44ef48c8f51695a4e42d8) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:58.117202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:58.118724 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:58.135793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 432/865] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:57 2022] GET /identity/v3/role_assignments?user.id=e74571f351a44ef48c8f51695a4e42d8&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:58.135793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:58.139778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:58.151411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee8f756-a906-4d1e-8f49-290621086ad4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:58.193381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:58.194245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.194621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.195002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.196041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:58.196462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:58.198046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:58.224272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 434/866] 10.222.0.16 () {56 vars in 1135 bytes} [Thu Jul 7 10:13:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:58.225347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:58.257751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73e96cc-0531-4f66-ae9f-80bf0e89eccb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:58.299261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:58.300005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.300285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.300571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.301297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:58.302741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:58.320727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 433/867] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:13:58 2022] POST /identity/v3/services => generated 269 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:13:58.344630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f8759-1983-43a5-8940-b30a4b83661a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:58.398985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a076d9de-7985-4936-86ab-51e86e8f5385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:58.399966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d9de-7985-4936-86ab-51e86e8f5385 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.400356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d9de-7985-4936-86ab-51e86e8f5385 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.400731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a076d9de-7985-4936-86ab-51e86e8f5385 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.417604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:58.479607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076d9de-7985-4936-86ab-51e86e8f5385 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:58.481211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a076d9de-7985-4936-86ab-51e86e8f5385 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:58.587930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 435/868] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:13:58 2022] GET /identity/v3/auth/tokens => generated 1261 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:13:58.594516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:58.650491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0656e-fa53-4191-97d2-d2e7cb37ac62 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.650821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0656e-fa53-4191-97d2-d2e7cb37ac62 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.651059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd0656e-fa53-4191-97d2-d2e7cb37ac62 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.652973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:58.653254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 436/869] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:13:58.659178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60326df-5388-47e6-829c-420efead819c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.659793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60326df-5388-47e6-829c-420efead819c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.660286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60326df-5388-47e6-829c-420efead819c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.680836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d60326df-5388-47e6-829c-420efead819c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:13:58.682020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d60326df-5388-47e6-829c-420efead819c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:13:58.710607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607f5215-23fc-410b-8feb-60498e1fe171 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:58.727432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:58.763924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 437/870] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:13:58 2022] POST /identity/v3/auth/tokens => generated 1261 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:13:58.764523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:58.764523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.764523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.764523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.774317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:58.776034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:58.783939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60326df-5388-47e6-829c-420efead819c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:13:58.795637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:58.796580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:58.797498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:13:58.797498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:13:58.797498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:13:58.797498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:13:58.801762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 434/871] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:13:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:58.802735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:58.891619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60326df-5388-47e6-829c-420efead819c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:58.916978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b5d298-34d9-4edf-a971-aa30045f8be8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:58.927518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:58.928480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:58.929328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.929795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.930227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.931112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:58.932660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:58.949836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 438/872] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:13:58 2022] POST /identity/v3/services => generated 251 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:13:58.970157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:58.971282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:58.971721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:58.972201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:58.973109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:58.973611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:58.975258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:58.990743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 435/873] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:58.996219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:59.043029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:59.113027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4b1ff-30d0-418a-ae78-fa2725ad766d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:59.147466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:59.148351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:59.148638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:59.148916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:59.156783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:59.158241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:13:59.174162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:13:59.175073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:13:59.176005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 439/874] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:13:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:13:59.183042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:59.292142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac70173-9a32-4d5d-86a9-7b766c338c0b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:59.327846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e52e45-8ad9-418b-99c2-e25da744339b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:59.328859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e52e45-8ad9-418b-99c2-e25da744339b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:59.329354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e52e45-8ad9-418b-99c2-e25da744339b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:59.329836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e52e45-8ad9-418b-99c2-e25da744339b None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:59.330840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e52e45-8ad9-418b-99c2-e25da744339b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:59.331385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e52e45-8ad9-418b-99c2-e25da744339b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:59.333052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e52e45-8ad9-418b-99c2-e25da744339b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:59.345508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 436/875] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:13:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:13:59.351522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:13:59.452995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:13:59.461281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab6ad46-157b-4a0e-87c5-313db521fc40 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:13:59.496654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f463aad6-a96d-4db1-baa2-cd1309a42b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:13:59.497404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463aad6-a96d-4db1-baa2-cd1309a42b29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:13:59.497684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463aad6-a96d-4db1-baa2-cd1309a42b29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:13:59.497962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f463aad6-a96d-4db1-baa2-cd1309a42b29 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:13:59.498792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463aad6-a96d-4db1-baa2-cd1309a42b29 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=3e2e080fd7144086ac2cf5628a7a68ed) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:13:59.499107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463aad6-a96d-4db1-baa2-cd1309a42b29 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:13:59.500584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f463aad6-a96d-4db1-baa2-cd1309a42b29 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:13:59.507092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 440/876] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:13:59 2022] GET /identity/v3/role_assignments?user.id=3e2e080fd7144086ac2cf5628a7a68ed&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:00.332828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd8515-8b53-4006-bf95-9dc8452e5c2e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.333383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd8515-8b53-4006-bf95-9dc8452e5c2e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.333383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd8515-8b53-4006-bf95-9dc8452e5c2e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.334936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 437/877] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:00.340221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721511d0-afef-477f-9b89-1f1e94c15cc9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.340609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721511d0-afef-477f-9b89-1f1e94c15cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.340845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721511d0-afef-477f-9b89-1f1e94c15cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.357139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-721511d0-afef-477f-9b89-1f1e94c15cc9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:00.358302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-721511d0-afef-477f-9b89-1f1e94c15cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:00.423904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 441/878] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:00.447327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd8515-8b53-4006-bf95-9dc8452e5c2e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:00.465651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:00.491811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7017ab-26ca-481e-9746-629099518447 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.492099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7017ab-26ca-481e-9746-629099518447 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.492371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7017ab-26ca-481e-9746-629099518447 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.493957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 442/879] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:00.509054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa91fd0-2b8e-4a55-9ebe-15f75d810b81 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.509759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa91fd0-2b8e-4a55-9ebe-15f75d810b81 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.510242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa91fd0-2b8e-4a55-9ebe-15f75d810b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.532521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2fa91fd0-2b8e-4a55-9ebe-15f75d810b81 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:00.534113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2fa91fd0-2b8e-4a55-9ebe-15f75d810b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:00.560659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd8515-8b53-4006-bf95-9dc8452e5c2e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:00.598372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:00.599636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.600189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.600671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.609905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 443/880] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:00.621229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:00.630154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:00.631089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:00.632361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 438/881] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jul 7 10:14:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:00.638378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa91fd0-2b8e-4a55-9ebe-15f75d810b81 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:00.640872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:00.659258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:00.724455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:00.750105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283772b-726a-448a-8c6e-732ff4a3e0fe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:00.756925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa91fd0-2b8e-4a55-9ebe-15f75d810b81 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:00.793412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:00.795781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.796275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.796713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.798215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:00.800071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:00.803166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:00.803969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.804354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.804771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.805580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:00.805990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:00.807435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:00.816479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:00.817591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:14:00.818601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:00.824284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 444/882] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jul 7 10:14:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:00.825651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:00.827161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 439/883] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jul 7 10:14:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:00.832054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:00.933497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:00.938057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbe357f-da5e-4fd0-9303-eb4826026088 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:00.964712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43effaf-afed-47fe-8f2e-303459998dde None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:00.974737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:00.975407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:00.975694 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:00.976018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:00.976731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:00.977042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:00.978408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:00.988540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 440/884] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:00.989859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:00.994292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:01.012594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.013731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.014236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.015231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.030563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.032329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:01.055689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:01.056662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:01.057533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:01.057533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:01.057533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:01.057533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:01.062506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 445/885] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:01.063894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.113408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4b85da-5767-4d85-b634-fc21d8622713 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.155229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc945cc-95cc-4e73-a13d-318cf352e210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.155939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc945cc-95cc-4e73-a13d-318cf352e210 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.156230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc945cc-95cc-4e73-a13d-318cf352e210 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.156509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc945cc-95cc-4e73-a13d-318cf352e210 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.157338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc945cc-95cc-4e73-a13d-318cf352e210 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:01.157652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc945cc-95cc-4e73-a13d-318cf352e210 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.159013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc945cc-95cc-4e73-a13d-318cf352e210 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.168844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742585a-1046-46eb-9fcd-461268b6f916 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.170110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 441/886] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:01.172197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bf589-9bd3-47d8-b7ad-5be2e6c9ac76 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.172197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bf589-9bd3-47d8-b7ad-5be2e6c9ac76 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.172390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91bf589-9bd3-47d8-b7ad-5be2e6c9ac76 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.173699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 442/887] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:01.174312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bf589-9bd3-47d8-b7ad-5be2e6c9ac76 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.204796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:01.211771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c3e647-797d-42d7-a7a2-f31e54d3804d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.211949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e647-797d-42d7-a7a2-f31e54d3804d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.212222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e647-797d-42d7-a7a2-f31e54d3804d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.212498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c3e647-797d-42d7-a7a2-f31e54d3804d None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.213221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3e647-797d-42d7-a7a2-f31e54d3804d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:01.213532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3e647-797d-42d7-a7a2-f31e54d3804d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.214903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c3e647-797d-42d7-a7a2-f31e54d3804d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.229180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 446/888] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:01.230577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587789-eab7-4df9-9e16-93f34eb9a385 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.230867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587789-eab7-4df9-9e16-93f34eb9a385 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.231145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf587789-eab7-4df9-9e16-93f34eb9a385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.247566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:01.250239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bf587789-eab7-4df9-9e16-93f34eb9a385 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:01.251897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bf587789-eab7-4df9-9e16-93f34eb9a385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:01.282474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91bf589-9bd3-47d8-b7ad-5be2e6c9ac76 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.316902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.317567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.317850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.318128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.318916 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] RBAC: Adding query filter params (interface=public, service_id=5acf9030209349328111c1ac8942567f, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:01.319229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.320628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.325335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 447/889] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:01.327786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 443/890] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/endpoints?service_id=5acf9030209349328111c1ac8942567f&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:01.330911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf587789-eab7-4df9-9e16-93f34eb9a385 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.331365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.435492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1a5838-7aef-4c5a-a315-f91eea9d3684 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.436457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf587789-eab7-4df9-9e16-93f34eb9a385 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.461520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:01.468650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:01.474498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.475208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.475468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.475826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.476422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.476626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.477284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.477570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.477845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.477940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.491792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 444/891] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/services => generated 1268 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:14:01.491792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.493936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.495426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:14:01.515666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:01.516869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:01.518152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 448/892] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:01.523456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.620252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d44b15-ea0e-40dd-9762-94be1985e8c0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.654389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325f543-5a18-4d81-aab0-ebf8495d2519 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.657640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd9dc14-fe48-4c71-b5c3-463e1bbf3f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.658455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9dc14-fe48-4c71-b5c3-463e1bbf3f2e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.658741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9dc14-fe48-4c71-b5c3-463e1bbf3f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.659014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd9dc14-fe48-4c71-b5c3-463e1bbf3f2e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.659672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9dc14-fe48-4c71-b5c3-463e1bbf3f2e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.661070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd9dc14-fe48-4c71-b5c3-463e1bbf3f2e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5fd9dc14-fe48-4c71-b5c3-463e1bbf3f2e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:01.674035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:01.675017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:01.675870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 445/893] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:01.680984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8805fe-d84b-485a-bace-e01cc9cc387f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.681294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8805fe-d84b-485a-bace-e01cc9cc387f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.681577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8805fe-d84b-485a-bace-e01cc9cc387f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.682904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 446/894] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:01.683606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8805fe-d84b-485a-bace-e01cc9cc387f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.698045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e251b779-3c01-4d23-a405-650125559fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.698721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251b779-3c01-4d23-a405-650125559fbb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.699008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251b779-3c01-4d23-a405-650125559fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.699290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e251b779-3c01-4d23-a405-650125559fbb None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.700723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251b779-3c01-4d23-a405-650125559fbb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:01.701039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251b779-3c01-4d23-a405-650125559fbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.702473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e251b779-3c01-4d23-a405-650125559fbb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.723865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 449/895] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:01.723865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fedb85-9ad5-434c-a292-ff8134d4804b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.723865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fedb85-9ad5-434c-a292-ff8134d4804b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.723865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fedb85-9ad5-434c-a292-ff8134d4804b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.744558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-25fedb85-9ad5-434c-a292-ff8134d4804b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:01.746093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25fedb85-9ad5-434c-a292-ff8134d4804b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:01.753739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:01.782339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:01.802264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8805fe-d84b-485a-bace-e01cc9cc387f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.823834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 450/896] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:01.823834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fedb85-9ad5-434c-a292-ff8134d4804b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.843255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.844097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.844503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.844878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.846181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:01.846856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:01.848642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:01.855682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 447/897] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:01.857177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:01.927237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fedb85-9ad5-434c-a292-ff8134d4804b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.963442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:01.964126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:01.964408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:01.964685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:01.968343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e640bf-e600-440a-9fe5-2a89fbd09ace None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:01.993631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:02.006925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.007773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.008053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.008333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.008954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:02.009070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.010488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:02.023117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:02.024250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:02.025105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 448/898] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:02.025896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.050464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.051947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.129938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0588162-76a4-4b41-8a90-ab9633618000 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.157470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 451/899] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:02.158311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.170283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.170955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.171300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.171582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.172319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:02.172625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.173975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.179094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 449/900] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:02.179682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.238825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:02.240329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:02.273949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2db823-ed59-4ce2-a0bd-27839ce879c4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.295341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf73667a-a2af-4bb9-982c-0c311a34ae02 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.316688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.317441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.317724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.318054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.318748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:02.319051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.320667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.332200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.332997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.333285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.333747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.333933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 452/901] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:02.334718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.347044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.348547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:02.366875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:02.367792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:02.368714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:02.368714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:02.368714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:02.368714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:02.368714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 450/902] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:02.369416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.463294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1675aa40-1f0f-4924-a9cc-891dcd458b7e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.487991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eef9358-0360-4792-8cff-ea68f9c9d057 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.490337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:02.505089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.505752 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.506037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.506313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.507100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] RBAC: Adding query filter params (interface=public, service_id=3b85a5b70dfc4ad99e6188baf410a108, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:02.507416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.508897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.510989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:02.519505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 453/903] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/endpoints?service_id=3b85a5b70dfc4ad99e6188baf410a108&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:02.520910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.536981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.537450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.537822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.538168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.538953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:02.539331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.540793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.546283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 451/904] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:02.552733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.648948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137e7599-c127-4f39-84ff-3f1b96ada0c8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.669403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1683a48c-ec01-4c82-845d-4e805b9c7c38 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.693741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.694855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.695328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.695941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.696879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:02.697396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.699178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.706663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.707664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.708171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.709364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.709364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.710479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.717925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 454/905] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:02.724146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 452/906] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/services => generated 1268 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:14:02.725968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.792917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:02.840472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35b222a-f7fa-4fd7-b74c-a4cbb5a1dfc4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:02.878277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:02.880430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:02.880714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:02.880988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:02.889234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:02.890681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:14:02.904937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:02.905851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:02.906734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 455/907] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:02.913267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:02.950827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:03.025368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2643ad13-9242-4529-bab8-8006ca683d68 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:03.062226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50d1c4-acfc-4295-ab27-4ceffc499670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:03.063267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50d1c4-acfc-4295-ab27-4ceffc499670 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:03.063792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50d1c4-acfc-4295-ab27-4ceffc499670 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:03.064293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50d1c4-acfc-4295-ab27-4ceffc499670 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:03.065280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50d1c4-acfc-4295-ab27-4ceffc499670 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:03.065789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50d1c4-acfc-4295-ab27-4ceffc499670 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:03.067418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50d1c4-acfc-4295-ab27-4ceffc499670 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:03.080840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 453/908] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:03.089685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:03.193767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2571a49d-d603-4fe0-b91b-99e7117ae550 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:03.211219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:03.228561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cd4fc-8cc9-4092-b98c-c255440f8dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:03.229220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd4fc-8cc9-4092-b98c-c255440f8dc7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:03.229496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd4fc-8cc9-4092-b98c-c255440f8dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:03.229758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd4fc-8cc9-4092-b98c-c255440f8dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:03.230581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd4fc-8cc9-4092-b98c-c255440f8dc7 None admin] RBAC: Adding query filter params (role.id=0504664beb9b4fd6a50b4b64828b071f, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=e74571f351a44ef48c8f51695a4e42d8) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:03.230902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd4fc-8cc9-4092-b98c-c255440f8dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:03.232922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd4fc-8cc9-4092-b98c-c255440f8dc7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:03.239146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 456/909] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:03 2022] GET /identity/v3/role_assignments?user.id=e74571f351a44ef48c8f51695a4e42d8&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=0504664beb9b4fd6a50b4b64828b071f => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:03.642968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231c47-5f52-4bd6-8652-44d528a41cd8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:03.643481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231c47-5f52-4bd6-8652-44d528a41cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:03.643559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab231c47-5f52-4bd6-8652-44d528a41cd8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:03.645189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 454/910] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:03.650284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0de634-c8e9-4cf1-901f-6bbfa3026139 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:03.650682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0de634-c8e9-4cf1-901f-6bbfa3026139 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:03.650846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0de634-c8e9-4cf1-901f-6bbfa3026139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:03.668503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3b0de634-c8e9-4cf1-901f-6bbfa3026139 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:03.670249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b0de634-c8e9-4cf1-901f-6bbfa3026139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:03.741620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 457/911] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:03.764589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231c47-5f52-4bd6-8652-44d528a41cd8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:03.862638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:03.874637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab231c47-5f52-4bd6-8652-44d528a41cd8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:03.910827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:03.911658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:03.911973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:03.912248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:03.912924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:03.914335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:03.928027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:03.929040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:03.929876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 455/912] 10.222.0.16 () {56 vars in 1132 bytes} [Thu Jul 7 10:14:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:03.937479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0de634-c8e9-4cf1-901f-6bbfa3026139 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:04.037476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0de634-c8e9-4cf1-901f-6bbfa3026139 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:04.050586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:04.071955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:04.072625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.072912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.073191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.073891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:04.074199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:04.076128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:04.081134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 458/913] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:04.086676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:04.189662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab5098c-5f2c-47ae-9d05-c9339e2bd92f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:04.224298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a261f-3b8f-4c90-92ba-7a68459544f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:04.224970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a261f-3b8f-4c90-92ba-7a68459544f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.225256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a261f-3b8f-4c90-92ba-7a68459544f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.225536 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a261f-3b8f-4c90-92ba-7a68459544f1 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.226255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a261f-3b8f-4c90-92ba-7a68459544f1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:04.226564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a261f-3b8f-4c90-92ba-7a68459544f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:04.228060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372a261f-3b8f-4c90-92ba-7a68459544f1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:04.233800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 456/914] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:04.239545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:04.344797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d50d20-6a15-436d-939a-c0faeac681b1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:04.380019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839130cd-b050-4a5e-880f-7e601374fdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:04.380680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839130cd-b050-4a5e-880f-7e601374fdc0 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.380957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839130cd-b050-4a5e-880f-7e601374fdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.381230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839130cd-b050-4a5e-880f-7e601374fdc0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.381969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839130cd-b050-4a5e-880f-7e601374fdc0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:04.383329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839130cd-b050-4a5e-880f-7e601374fdc0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:04.405084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 459/915] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:14:04 2022] POST /identity/v3/endpoints => generated 347 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:04.704216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a08d27-abb1-451f-b1ee-9e4c077494f5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.704831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a08d27-abb1-451f-b1ee-9e4c077494f5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.704831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a08d27-abb1-451f-b1ee-9e4c077494f5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.706330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 457/916] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:04.711849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027a8c8-a30d-4d76-9f36-e3b5bd9f9741 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.712271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027a8c8-a30d-4d76-9f36-e3b5bd9f9741 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.712446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b027a8c8-a30d-4d76-9f36-e3b5bd9f9741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.730174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b027a8c8-a30d-4d76-9f36-e3b5bd9f9741 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:04.731398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b027a8c8-a30d-4d76-9f36-e3b5bd9f9741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:04.758129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:04.807825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 460/917] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:04.831164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a08d27-abb1-451f-b1ee-9e4c077494f5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:04.854227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:04.940001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a08d27-abb1-451f-b1ee-9e4c077494f5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:04.976080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:04.976797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.977091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.977263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316a6ac-987e-453c-a3c6-3ca62bce387e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.977368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.977731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316a6ac-987e-453c-a3c6-3ca62bce387e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.978109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d316a6ac-987e-453c-a3c6-3ca62bce387e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.978223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:04.979485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:04.980472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 461/918] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:04.986084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9d6a8-5363-4504-b7f3-e6e0ae3bb690 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:04.986562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9d6a8-5363-4504-b7f3-e6e0ae3bb690 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:04.986977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f9d6a8-5363-4504-b7f3-e6e0ae3bb690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:04.996029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 458/919] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:14:04 2022] POST /identity/v3/services => generated 241 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:05.011059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b6f9d6a8-5363-4504-b7f3-e6e0ae3bb690 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:05.013223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b6f9d6a8-5363-4504-b7f3-e6e0ae3bb690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:05.097936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 462/920] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:05.120346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:05.213897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:05.240038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b0838b-813f-4652-9d8c-05ea5dcb2616 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:05.276024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:05.276830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.277112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.277388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.278084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:05.279445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:05.289587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec3f6-5259-4ced-a459-478837323642 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.289873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec3f6-5259-4ced-a459-478837323642 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.290154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ec3f6-5259-4ced-a459-478837323642 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:05.291944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:05.292974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 07 10:14:05.294403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:05.295184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 463/921] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:05.295264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 459/922] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:14:05 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:05.297476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a213b9-6269-48eb-a282-4225c150e24e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.297768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a213b9-6269-48eb-a282-4225c150e24e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.298049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a213b9-6269-48eb-a282-4225c150e24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.301428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:05.323549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-41a213b9-6269-48eb-a282-4225c150e24e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:05.325771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-41a213b9-6269-48eb-a282-4225c150e24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:05.406572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 464/923] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:05.421348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f29e2b9-418e-4ff2-afea-b0744714b866 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:05.429376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a213b9-6269-48eb-a282-4225c150e24e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:05.445715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:05.470470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:05.471476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.471991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.472497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.473494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:05.474011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:05.476129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:05.484516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 460/924] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:14:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:05.490095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:05.505133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:05.563397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a213b9-6269-48eb-a282-4225c150e24e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:05.600994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:05.602145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.602623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.603079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.613896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:05.616012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9ec0a-6413-4484-8e5f-2447651084bd None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:05.623147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:05.624105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:05.624972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:05.627656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 465/925] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jul 7 10:14:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:05.634401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:05.659466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:05.660099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.660387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.660666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.661381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:05.661752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:05.663181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:05.674475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 461/926] 10.222.0.16 () {56 vars in 1114 bytes} [Thu Jul 7 10:14:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:05.675363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:05.721030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:05.748492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41cd3c0-72d5-41a4-a772-8ed7dfcfbd8e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:05.751668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:05.784380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f479e-7715-4709-9e3a-2754088001ad None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:05.784748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:05.785631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.786117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.786591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.787529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:05.788067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:05.789648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:05.798513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 466/927] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jul 7 10:14:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:05.804847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:05.829250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:05.830457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.830929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.831642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:05.833021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:05.834583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:05.868474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 462/928] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:14:05 2022] POST /identity/v3/endpoints => generated 346 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:05.937628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ef85d-89a3-48df-8f9e-1de6ce283fd6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:05.977195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:05.983446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:05.984291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:05.984575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:05.984852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.001432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.003285 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:06.022368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:06.023344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:06.024246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:06.024246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:06.024246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:06.024246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:06.026615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 467/929] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:06.032384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:06.140852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6bb244-bd7f-464f-bb48-8c93f987c89f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:06.168030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:06.176772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:06.177762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.178264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.178744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.179693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:06.180278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.181810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.200449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 463/930] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:06.202573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:06.306778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a36d2-c57b-492d-8cdf-1bd6febd31fb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:06.342437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4df7800-4dd8-446f-a096-ddc1cab73549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:06.343525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df7800-4dd8-446f-a096-ddc1cab73549 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.343998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df7800-4dd8-446f-a096-ddc1cab73549 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.344422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4df7800-4dd8-446f-a096-ddc1cab73549 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.352645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df7800-4dd8-446f-a096-ddc1cab73549 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.354310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4df7800-4dd8-446f-a096-ddc1cab73549 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b4df7800-4dd8-446f-a096-ddc1cab73549 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:14:06.369408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:06.370575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:14:06.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:06.378872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 468/931] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:14:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:06.380496 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:06.492010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb40b98c-7521-4c77-b7e9-c0b878c095db None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:06.497935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de273cd-4910-4b48-aa1c-7e72559b066a None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.498339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de273cd-4910-4b48-aa1c-7e72559b066a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.498719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de273cd-4910-4b48-aa1c-7e72559b066a None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.500447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 469/932] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:06.505746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b40084-901a-45f8-ae77-a0c5d1a11f27 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.506146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b40084-901a-45f8-ae77-a0c5d1a11f27 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.506528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b40084-901a-45f8-ae77-a0c5d1a11f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.532793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-24b40084-901a-45f8-ae77-a0c5d1a11f27 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:06.535185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24b40084-901a-45f8-ae77-a0c5d1a11f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:06.549142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:06.549908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.550361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.550705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.551736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:06.552144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.553651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.574559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 464/933] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jul 7 10:14:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:06.580665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:06.585924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:06.622189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 470/934] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:06.640291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:06.643704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b40084-901a-45f8-ae77-a0c5d1a11f27 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:06.710944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc880eb-7eba-4f5b-9552-6cac233ec63a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:06.747687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:06.748448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.748730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.749002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.760299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.761741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.769958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b40084-901a-45f8-ae77-a0c5d1a11f27 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:06.780610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:06.781520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:06.782425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:06.782425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:06.782425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:06.782425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:06.786623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 465/935] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:06.788013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:06.816421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:06.817255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.817539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.817815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] PATH_INFO: `/v3/services/compute` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.818469 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.819864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:06.832101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:06.833125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:06.833976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:06.838173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 471/936] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:06.839411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:06.882067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:06.928287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:06.934729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfb2b71-8226-4211-8a3d-ae00572ce5fa None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:06.951295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5e134a-4538-4340-b948-1ea04eabb26e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:06.977521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:06.978205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.978494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.978777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.979511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:06.979865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.982502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.989724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:06.990711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:06.991199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:06.991817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:06.992802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:06.993322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:06.996016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:06.996151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 466/937] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:07.001891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.003550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 472/938] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:14:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:07.009878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.115221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96afab5c-d164-410b-bea5-188c76862281 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.116969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:07.118236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6794dba-37b3-4f17-89a2-ae15e50f7709 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.151656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b211ba-45ab-4b3d-b3dc-dbfe9ed61410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.152451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b211ba-45ab-4b3d-b3dc-dbfe9ed61410 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.153333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b211ba-45ab-4b3d-b3dc-dbfe9ed61410 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.153620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b211ba-45ab-4b3d-b3dc-dbfe9ed61410 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.154332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b211ba-45ab-4b3d-b3dc-dbfe9ed61410 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:07.154646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b211ba-45ab-4b3d-b3dc-dbfe9ed61410 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:07.155731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.156042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b211ba-45ab-4b3d-b3dc-dbfe9ed61410 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:07.156559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.156843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.157192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.160992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 473/939] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:14:07 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:07.163916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97e410-6b31-4bb5-9767-e8053e7bcb59 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.163916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97e410-6b31-4bb5-9767-e8053e7bcb59 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.163916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f97e410-6b31-4bb5-9767-e8053e7bcb59 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.164278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 474/940] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:07.165462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:07.166945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:07.173996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a037903-e41c-4808-9b1a-8bb612dc4bb2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.174308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a037903-e41c-4808-9b1a-8bb612dc4bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.174561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a037903-e41c-4808-9b1a-8bb612dc4bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:14:07.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:07.187610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:07.188555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:07.192913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 467/941] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:07.193391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9a037903-e41c-4808-9b1a-8bb612dc4bb2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:07.194350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.194459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9a037903-e41c-4808-9b1a-8bb612dc4bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:07.215827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:07.280253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 475/942] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:07.297637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a037903-e41c-4808-9b1a-8bb612dc4bb2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.309216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d5e265-94f3-4808-bea8-2fcca2d2146f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.351570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d72b-2397-4867-a88d-823638239437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.352266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d72b-2397-4867-a88d-823638239437 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.352553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d72b-2397-4867-a88d-823638239437 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.352831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9531d72b-2397-4867-a88d-823638239437 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.353617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531d72b-2397-4867-a88d-823638239437 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:07.353926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531d72b-2397-4867-a88d-823638239437 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:07.355353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9531d72b-2397-4867-a88d-823638239437 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:07.368777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 468/943] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:14:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:07.374829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d72b-2397-4867-a88d-823638239437 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.412501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:07.420932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a037903-e41c-4808-9b1a-8bb612dc4bb2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.457496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.458713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.459235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.459724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.460608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:07.462181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:07.475360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:07.476168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:07.477181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:07.478078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:07.481142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 476/944] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:07.486565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.509569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9531d72b-2397-4867-a88d-823638239437 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.552068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.552986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.553515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.553741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/e74571f351a44ef48c8f51695a4e42d8/roles/0504664beb9b4fd6a50b4b64828b071f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.556292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=e74571f351a44ef48c8f51695a4e42d8, role_id=0504664beb9b4fd6a50b4b64828b071f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:07.557860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:07.579848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 469/945] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:14:07 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/e74571f351a44ef48c8f51695a4e42d8/roles/0504664beb9b4fd6a50b4b64828b071f => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:14:07.626102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ef8b6-477c-4a65-b38c-bad53e493e37 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.673683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.674392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.674680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.674957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.675687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:07.676035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:07.677457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:07.683501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 477/946] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:07.689587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.801021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff8588a-7055-4c7d-853e-419a83382f46 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.836215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.836889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.837225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.837507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.838223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:07.838529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:07.839915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:07.845751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 470/947] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:07.851538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:07.879537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:07.960606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de516e4b-1711-4b80-841c-881a86d46434 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:07.996880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b9dd70-4fd2-4489-8443-e27977475073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:07.997546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9dd70-4fd2-4489-8443-e27977475073 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:07.997834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9dd70-4fd2-4489-8443-e27977475073 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:07.998117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b9dd70-4fd2-4489-8443-e27977475073 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:07.998912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9dd70-4fd2-4489-8443-e27977475073 None admin] RBAC: Adding query filter params (interface=public, service_id=e69da2a4ecd44972836a6fa7bc36b398, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:07.999227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9dd70-4fd2-4489-8443-e27977475073 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:08.000643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b9dd70-4fd2-4489-8443-e27977475073 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:08.006311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 478/948] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:14:07 2022] GET /identity/v3/endpoints?service_id=e69da2a4ecd44972836a6fa7bc36b398&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:08.011809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:08.014803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:08.049390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2434a9-bbcf-421f-a940-873415313994 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:08.049779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2434a9-bbcf-421f-a940-873415313994 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:08.050154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2434a9-bbcf-421f-a940-873415313994 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:08.056706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 479/949] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:08.058301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57e283-5525-459d-a917-85cf5d2414c0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:08.058681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57e283-5525-459d-a917-85cf5d2414c0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:08.059090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df57e283-5525-459d-a917-85cf5d2414c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:08.076443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-df57e283-5525-459d-a917-85cf5d2414c0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:08.077739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-df57e283-5525-459d-a917-85cf5d2414c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:08.128599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cfa63-020a-457f-8d5b-578649330190 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:08.164877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:08.165535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:08.165863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:08.166142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:08.166862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:08.166992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 480/950] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:08.176286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:08.185641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 471/951] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:14:08 2022] GET /identity/v3/services => generated 1497 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:14:08.195481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57e283-5525-459d-a917-85cf5d2414c0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:08.223426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:08.306471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df57e283-5525-459d-a917-85cf5d2414c0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:08.349904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:08.351035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:08.351530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:08.352048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:08.353034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:08.354773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:08.356544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING keystone.server.flask.common [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 07 10:14:08.375989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.sql.core [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 07 10:14:08.375989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 07 10:14:08.375989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [parameters: {'id': '68b33f2788af43b18541c5027a765ee9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 07 10:14:08.375989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80141) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 68b33f2788af43b18541c5027a765ee9.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 68b33f2788af43b18541c5027a765ee9. Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 10:14:08.384753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 07 10:14:08.385579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application yield resource Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application next(self.gen) Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.rollback() Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.force_reraise() Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise self.value Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application session.commit() Jul 07 10:14:08.386582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._prepare_impl() Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.session.flush() Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._flush(objects) Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application compat.raise_( Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application flush_context.execute() Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 07 10:14:08.388047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rec.execute(self) Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 07 10:14:08.388965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application conn.query(q) Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application result.read() Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 10:14:08.389763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [parameters: {'id': '68b33f2788af43b18541c5027a765ee9', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:08.391830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:08.393739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 68b33f2788af43b18541c5027a765ee9. Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:08.396234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 481/952] 10.222.0.16 () {60 vars in 1140 bytes} [Thu Jul 7 10:14:08 2022] POST /identity/v3/projects => generated 262 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:08.399052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:08.470794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:08.506553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d9e402-c5a2-4906-bd07-7271bfc69315 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:08.541707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a9f579-cdd4-44dd-ae16-bc54e7786162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:08.542510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9f579-cdd4-44dd-ae16-bc54e7786162 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:08.542796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9f579-cdd4-44dd-ae16-bc54e7786162 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:08.543078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a9f579-cdd4-44dd-ae16-bc54e7786162 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:08.550491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9f579-cdd4-44dd-ae16-bc54e7786162 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:08.551961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a9f579-cdd4-44dd-ae16-bc54e7786162 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-80a9f579-cdd4-44dd-ae16-bc54e7786162 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:14:08.567169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:08.568119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:08.569749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 472/953] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:08.574586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:08.656774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:08.684572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc45649c-108c-4224-bf09-3a6e06690131 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:08.724919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d367b4e-4651-4957-ad5a-df49220ed683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:08.725580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d367b4e-4651-4957-ad5a-df49220ed683 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:08.725863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d367b4e-4651-4957-ad5a-df49220ed683 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:08.726140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d367b4e-4651-4957-ad5a-df49220ed683 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:08.726891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d367b4e-4651-4957-ad5a-df49220ed683 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:08.727199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d367b4e-4651-4957-ad5a-df49220ed683 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:08.728875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d367b4e-4651-4957-ad5a-df49220ed683 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:08.750880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 482/954] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:14:08 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:09.296890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2769c2-5710-484e-92c9-0a1a04c25269 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:09.297624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2769c2-5710-484e-92c9-0a1a04c25269 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:09.297624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2769c2-5710-484e-92c9-0a1a04c25269 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:09.299088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 473/955] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:09.304097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6922b13-a237-445f-9b8a-ec9233a307a1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:09.304572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6922b13-a237-445f-9b8a-ec9233a307a1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:09.304744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6922b13-a237-445f-9b8a-ec9233a307a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:09.322896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f6922b13-a237-445f-9b8a-ec9233a307a1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:09.324481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6922b13-a237-445f-9b8a-ec9233a307a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:09.398160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 483/956] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:09.421583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2769c2-5710-484e-92c9-0a1a04c25269 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:09.529260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2769c2-5710-484e-92c9-0a1a04c25269 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:09.555419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:09.564270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca146f8-b0e0-4455-8890-c5cb5e353bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:09.565046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca146f8-b0e0-4455-8890-c5cb5e353bf1 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:09.565541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca146f8-b0e0-4455-8890-c5cb5e353bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:09.565888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca146f8-b0e0-4455-8890-c5cb5e353bf1 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:09.566773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca146f8-b0e0-4455-8890-c5cb5e353bf1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:09.568366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca146f8-b0e0-4455-8890-c5cb5e353bf1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:09.585599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 474/957] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:14:09 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:09.765822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcc7fe-af6b-4e23-8486-76c0055f2e15 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:09.765822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcc7fe-af6b-4e23-8486-76c0055f2e15 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:09.765822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bcc7fe-af6b-4e23-8486-76c0055f2e15 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:09.766557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 484/958] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:09.771983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7773dd-fce2-408c-8d72-9854e0d69fab None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:09.772349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7773dd-fce2-408c-8d72-9854e0d69fab None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:09.772702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7773dd-fce2-408c-8d72-9854e0d69fab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:09.791134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ab7773dd-fce2-408c-8d72-9854e0d69fab None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:09.792413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ab7773dd-fce2-408c-8d72-9854e0d69fab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:09.869092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 475/959] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:09.893050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcc7fe-af6b-4e23-8486-76c0055f2e15 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:09.966291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:10.015941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bcc7fe-af6b-4e23-8486-76c0055f2e15 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.051123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:10.051974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.052260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.052537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.063385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:10.071637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:10.072631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:10.073909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 485/960] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jul 7 10:14:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:10.081600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7773dd-fce2-408c-8d72-9854e0d69fab None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.190650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7773dd-fce2-408c-8d72-9854e0d69fab None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.204552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:10.231710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f17341-8d0c-498e-b2b5-f36496537c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:10.232751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f17341-8d0c-498e-b2b5-f36496537c87 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.233240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f17341-8d0c-498e-b2b5-f36496537c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.233745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f17341-8d0c-498e-b2b5-f36496537c87 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.234712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f17341-8d0c-498e-b2b5-f36496537c87 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:10.235222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f17341-8d0c-498e-b2b5-f36496537c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:10.236827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f17341-8d0c-498e-b2b5-f36496537c87 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:10.250893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 476/961] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:10.252516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.354478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd090c84-5ecc-4338-8bb0-265153cafe08 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.380529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:10.388844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c073b2-2437-4984-83e1-e3e2872e3300 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.389321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c073b2-2437-4984-83e1-e3e2872e3300 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.389423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c073b2-2437-4984-83e1-e3e2872e3300 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.390890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 477/962] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:10.395776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:10.396096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d50d-58a3-481e-a59b-90540fce1473 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.396382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d50d-58a3-481e-a59b-90540fce1473 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.396661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac80d50d-58a3-481e-a59b-90540fce1473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.397476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.398070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.398644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.410568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:10.412368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:10.413704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ac80d50d-58a3-481e-a59b-90540fce1473 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:10.415012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ac80d50d-58a3-481e-a59b-90540fce1473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:10.431051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:10.432298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:10.433199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:10.433199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:10.433199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:10.433199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:10.436102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 486/963] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:10.442279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.492068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 478/964] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:10.513893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80d50d-58a3-481e-a59b-90540fce1473 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.559160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fad633-188e-46df-b2af-5bae583a0ad8 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.596106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:10.597010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.597470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.598003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.598873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:10.599333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:10.600968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:10.614578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 487/965] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:10.623787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.625090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac80d50d-58a3-481e-a59b-90540fce1473 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.628740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:10.674039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:10.676615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:10.677940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.678426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.678955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.679865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:10.683375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:10.702066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:10.703163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:10.704066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:10.705670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 479/966] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:10.711466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.765729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9463d298-7360-4978-8e4b-d329006f87a2 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.816896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:10.818339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.818841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.819334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb55a44-e4c8-40d7-8da8-e22c6dc43d24 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.819686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.830008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:10.832398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:14:10.854455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:10.855463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 07 10:14:10.857004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:10.859190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 488/967] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:10.859309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:10.859309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:10.859580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:10.863312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:10.863312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:10.863312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:10.863312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:10.864930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.869776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 480/968] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:10.874347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:10.914138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:10.989272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:10.993633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52749a3-fdd0-492c-a053-6eb66cc66e58 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:10.995795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c451d7-62da-4446-a7e9-905c25dec853 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.028650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea28202-d060-41c0-9ca8-9754db420785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.029317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea28202-d060-41c0-9ca8-9754db420785 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.029605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea28202-d060-41c0-9ca8-9754db420785 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.029885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea28202-d060-41c0-9ca8-9754db420785 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.030587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea28202-d060-41c0-9ca8-9754db420785 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:11.030897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea28202-d060-41c0-9ca8-9754db420785 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.032064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687f4862-a468-4914-8e46-8a349aafd8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.032276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea28202-d060-41c0-9ca8-9754db420785 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.032736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4862-a468-4914-8e46-8a349aafd8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.033022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4862-a468-4914-8e46-8a349aafd8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.033499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687f4862-a468-4914-8e46-8a349aafd8d9 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.034661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f4862-a468-4914-8e46-8a349aafd8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:11.034981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f4862-a468-4914-8e46-8a349aafd8d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.036586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687f4862-a468-4914-8e46-8a349aafd8d9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.041729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 481/969] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:11.043690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b793-8ac0-49d0-b078-1e6d5e742662 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.044017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b793-8ac0-49d0-b078-1e6d5e742662 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.044307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0269b793-8ac0-49d0-b078-1e6d5e742662 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.047784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 482/970] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:11.047784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b793-8ac0-49d0-b078-1e6d5e742662 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:11.058458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 489/971] 10.222.0.16 () {56 vars in 1149 bytes} [Thu Jul 7 10:14:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:11.059962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9d462-e576-46a9-ae64-5a128746a612 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.060259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9d462-e576-46a9-ae64-5a128746a612 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.060529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d9d462-e576-46a9-ae64-5a128746a612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.077036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d1d9d462-e576-46a9-ae64-5a128746a612 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:11.078224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d1d9d462-e576-46a9-ae64-5a128746a612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:11.147808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 490/972] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:11.147808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9d462-e576-46a9-ae64-5a128746a612 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:11.156191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:11.164986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0269b793-8ac0-49d0-b078-1e6d5e742662 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.200396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.201066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.201358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.201639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.202380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.203772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.231994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 483/973] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:14:11 2022] POST /identity/v3/endpoints => generated 349 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:11.232972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:11.249247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:11.270051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d9d462-e576-46a9-ae64-5a128746a612 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.319557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.320456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.320752 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.321027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.355410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d7e88-b77e-493e-9594-1592084cca48 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.394596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.395339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.395569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.395910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.396621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.398239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.399576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING keystone.server.flask.common [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 07 10:14:11.400568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.401738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.413405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.sql.core [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 07 10:14:11.413405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 07 10:14:11.413405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [parameters: {'user_id': 'd1760a995d224ece8fbd231231225dcb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 07 10:14:11.413405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80142) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 07 10:14:11.429995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 10:14:11.430959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application yield resource Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application next(self.gen) Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.rollback() Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.force_reraise() Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise self.value Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application session.commit() Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 07 10:14:11.432424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._prepare_impl() Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.session.flush() Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._flush(objects) Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application compat.raise_( Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application flush_context.execute() Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rec.execute(self) Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 07 10:14:11.433610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 07 10:14:11.434878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result = self._query(query) Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application conn.query(q) Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application result.read() Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 07 10:14:11.436018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [parameters: {'user_id': 'd1760a995d224ece8fbd231231225dcb', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:11.438631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 07 10:14:11.440723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 484/974] 10.222.0.16 () {60 vars in 1131 bytes} [Thu Jul 7 10:14:11 2022] POST /identity/v3/users => generated 160 bytes in 199 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:11.444577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:11.462289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:11.504218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:11.541941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 491/975] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:11 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:11.544338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:11.578079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b7d43e-2496-437b-95cb-4391e1d21c27 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.627055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.628514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.629288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.629754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.641083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.643552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.651815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84eb09b-7b4e-4d89-be23-ba406fd007e5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:14:11.662142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:11.663104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 07 10:14:11.664110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:11.669986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 485/976] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:14:11 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:11.671175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:11.694861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.695953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.696382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.696804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.711999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.714543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:11.738715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:11.739956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:11.741062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:11.741062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:11.741062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:11.741062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:11.747831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 492/977] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:11.751184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:11.763612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:11.783395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:11.792131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dc85e2-550c-46eb-b861-bdffc9d9ccb3 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.833162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2adcd2-47fd-4508-8639-1817c122b3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.833857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2adcd2-47fd-4508-8639-1817c122b3b3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.834145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2adcd2-47fd-4508-8639-1817c122b3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.834423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2adcd2-47fd-4508-8639-1817c122b3b3 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.835589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2adcd2-47fd-4508-8639-1817c122b3b3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:11.835928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2adcd2-47fd-4508-8639-1817c122b3b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.837668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2adcd2-47fd-4508-8639-1817c122b3b3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.857096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc97efb-8d5c-4375-a632-8a72ab688a74 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:11.882461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 486/978] 10.222.0.16 () {56 vars in 1101 bytes} [Thu Jul 7 10:14:11 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:11.882461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45c3f3-3c3e-463e-8229-1d2d9394776d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.882461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45c3f3-3c3e-463e-8229-1d2d9394776d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.882461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45c3f3-3c3e-463e-8229-1d2d9394776d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.882461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 487/979] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:11.884886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3afd2-af17-43a8-b5a2-9ec972f62eda None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.885024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3afd2-af17-43a8-b5a2-9ec972f62eda None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.885475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3afd2-af17-43a8-b5a2-9ec972f62eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.905861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b1d3afd2-af17-43a8-b5a2-9ec972f62eda None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:11.907440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:11.907582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1d3afd2-af17-43a8-b5a2-9ec972f62eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:11.908379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:11.908782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:11.909131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:11.910295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:11.910762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:11.912338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:11.926988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 493/980] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:11.932204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.001431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 488/981] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:12.024510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3afd2-af17-43a8-b5a2-9ec972f62eda None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.064763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:12.075779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd54cc3a-4375-4900-961b-bb6b6b7320bd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.105084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:12.129725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.130818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.131105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.131386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.144931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.146706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.157441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3afd2-af17-43a8-b5a2-9ec972f62eda None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:14:12.170688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:12.171615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:12.172542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 494/982] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:12.176698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.206514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.207367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.207656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.208011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.208609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.210037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:12.222218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:12.223203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:12.224118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 489/983] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:12.229290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.286139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b4f39e-c515-43a5-96ae-4f9b1c0b4351 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.320395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.321061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.321351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.321628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.322411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:12.322725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.324679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.336733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 495/984] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:14:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:12.337622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dacc69-5a8c-4118-91bd-557cc495a722 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.343517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.358947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:12.368090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:12.378214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.378794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.379152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.379422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.380087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:12.380398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.381766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.392092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 490/985] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:14:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:12.393094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.460047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba3e15e-d3ab-4c7a-ab40-3cf6cc0e451e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.494340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.494996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.495280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.495558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.496933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] RBAC: Adding query filter params (role.id=0504664beb9b4fd6a50b4b64828b071f, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=e74571f351a44ef48c8f51695a4e42d8) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:12.497250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.498670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.501477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7324b66-f79d-4960-876d-58396889f5cc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.507693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 496/986] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:12 2022] GET /identity/v3/role_assignments?user.id=e74571f351a44ef48c8f51695a4e42d8&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=0504664beb9b4fd6a50b4b64828b071f => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:12.562545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.563155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.563440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.563717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.564472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:12.564780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.566181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.572302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 491/987] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:14:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:12.578530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.661921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:12.687909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceadef2a-3040-4f5a-9b8e-c1a6e60480f7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.721772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42252734-cb46-4cc3-84b1-a93f0d133f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.722435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252734-cb46-4cc3-84b1-a93f0d133f43 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.722718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252734-cb46-4cc3-84b1-a93f0d133f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.722990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42252734-cb46-4cc3-84b1-a93f0d133f43 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.723785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42252734-cb46-4cc3-84b1-a93f0d133f43 None admin] RBAC: Adding query filter params (interface=public, service_id=fd2f86f97e4a4a2ab9d814a827068308, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:12.724102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42252734-cb46-4cc3-84b1-a93f0d133f43 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.725753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42252734-cb46-4cc3-84b1-a93f0d133f43 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.731448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 497/988] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:14:12 2022] GET /identity/v3/endpoints?service_id=fd2f86f97e4a4a2ab9d814a827068308&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:12.737308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:12.793897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:12.839653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9898b9-2b46-450c-a044-03f4c4da0699 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:12.873489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8475f8b-bb62-437e-bca2-961ecd9e760b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:12.874162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8475f8b-bb62-437e-bca2-961ecd9e760b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:12.874448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8475f8b-bb62-437e-bca2-961ecd9e760b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:12.874727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8475f8b-bb62-437e-bca2-961ecd9e760b None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:12.875408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8475f8b-bb62-437e-bca2-961ecd9e760b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:12.876827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8475f8b-bb62-437e-bca2-961ecd9e760b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:12.887828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 492/989] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:14:12 2022] GET /identity/v3/services => generated 1727 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:14:13.351021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1ba8-70e3-48da-b8ac-ce46cf902ffb None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:13.351394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1ba8-70e3-48da-b8ac-ce46cf902ffb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:13.351594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7f1ba8-70e3-48da-b8ac-ce46cf902ffb None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:13.353121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 498/990] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:13.358116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeae9d-d3dd-4f87-a1b2-ba4d7e2eb4a3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:13.358417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeae9d-d3dd-4f87-a1b2-ba4d7e2eb4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:13.358703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aeae9d-d3dd-4f87-a1b2-ba4d7e2eb4a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:13.375784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-15aeae9d-d3dd-4f87-a1b2-ba4d7e2eb4a3 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:13.376976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-15aeae9d-d3dd-4f87-a1b2-ba4d7e2eb4a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:13.443761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 493/991] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:13.466483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f1ba8-70e3-48da-b8ac-ce46cf902ffb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:13.577441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7f1ba8-70e3-48da-b8ac-ce46cf902ffb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:13.611357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:13.612111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:13.612392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:13.612663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:13.613300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:13.614680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:13.627162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:13.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:13.629339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 499/992] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:13.634275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeae9d-d3dd-4f87-a1b2-ba4d7e2eb4a3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:13.686740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:13.738040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aeae9d-d3dd-4f87-a1b2-ba4d7e2eb4a3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:13.771421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:13.772115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:13.772407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:13.772688 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:13.773387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:13.773703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:13.775069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:13.784413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 494/993] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:13.785628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:13.789513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:13.887682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778262d0-68fb-4528-b8c8-b28529883604 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:13.923180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:13.923885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:13.924172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:13.924449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:13.925149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:13.925459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:13.926828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:13.931930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 500/994] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:13.937817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:14.039760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c685c5-4c7d-481f-b45d-12cabc1d12b5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:14.054417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:14.077066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac28f517-642b-4866-8a32-368d5e74fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:14.077952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28f517-642b-4866-8a32-368d5e74fe75 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.078429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28f517-642b-4866-8a32-368d5e74fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.078849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac28f517-642b-4866-8a32-368d5e74fe75 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.079812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28f517-642b-4866-8a32-368d5e74fe75 None admin] RBAC: Adding query filter params (interface=admin, service_id=e69da2a4ecd44972836a6fa7bc36b398, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:14.080265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28f517-642b-4866-8a32-368d5e74fe75 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:14.081757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac28f517-642b-4866-8a32-368d5e74fe75 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:14.087398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 495/995] 10.222.0.16 () {56 vars in 1249 bytes} [Thu Jul 7 10:14:13 2022] GET /identity/v3/endpoints?service_id=e69da2a4ecd44972836a6fa7bc36b398&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:14.093151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:14.162543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:14.197106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01017133-46a3-4b2a-98d5-3670f97ca25a None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:14.231809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e17a46-6515-4d21-b5b1-54c5a500903b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:14.232470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17a46-6515-4d21-b5b1-54c5a500903b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.232749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17a46-6515-4d21-b5b1-54c5a500903b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.233021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e17a46-6515-4d21-b5b1-54c5a500903b None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.233725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e17a46-6515-4d21-b5b1-54c5a500903b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:14.235290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e17a46-6515-4d21-b5b1-54c5a500903b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:14.247786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 501/996] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:14:14 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:14:14.575737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3600-cb0d-4bb3-ac33-66f6ca94cbcb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.576071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3600-cb0d-4bb3-ac33-66f6ca94cbcb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.576330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580b3600-cb0d-4bb3-ac33-66f6ca94cbcb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.577750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 496/997] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:14.582878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f1423-31f0-414f-9cf3-d62b30b59c01 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.583311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f1423-31f0-414f-9cf3-d62b30b59c01 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.583475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f1423-31f0-414f-9cf3-d62b30b59c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.600184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c77f1423-31f0-414f-9cf3-d62b30b59c01 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:14.601371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c77f1423-31f0-414f-9cf3-d62b30b59c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:14.668950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 502/998] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:14.692253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b3600-cb0d-4bb3-ac33-66f6ca94cbcb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:14.795478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580b3600-cb0d-4bb3-ac33-66f6ca94cbcb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:14.829535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6968d5dd-fb8f-4d2c-862a-5e88559916e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:14.830332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968d5dd-fb8f-4d2c-862a-5e88559916e8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.830617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968d5dd-fb8f-4d2c-862a-5e88559916e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.830891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6968d5dd-fb8f-4d2c-862a-5e88559916e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.831541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968d5dd-fb8f-4d2c-862a-5e88559916e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:14.832920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6968d5dd-fb8f-4d2c-862a-5e88559916e8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-6968d5dd-fb8f-4d2c-862a-5e88559916e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:14.844943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:14.845889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:14.846727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:14.851122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 497/999] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jul 7 10:14:14 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:14.852200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f1423-31f0-414f-9cf3-d62b30b59c01 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:14.912843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:14.954007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f1423-31f0-414f-9cf3-d62b30b59c01 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:14.966111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe7483-61a1-4cb5-a8ec-a1a0f37be106 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.966414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe7483-61a1-4cb5-a8ec-a1a0f37be106 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.966693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe7483-61a1-4cb5-a8ec-a1a0f37be106 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.968247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 498/1000] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:14.973078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bbf8c-4cd9-4d43-b437-7f5d38399716 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.973369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bbf8c-4cd9-4d43-b437-7f5d38399716 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.973652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bbf8c-4cd9-4d43-b437-7f5d38399716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.990310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d25bbf8c-4cd9-4d43-b437-7f5d38399716 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:14.991507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d25bbf8c-4cd9-4d43-b437-7f5d38399716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:14.994607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:14.995274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:14.995554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:14.995926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:14.996627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] RBAC: Adding query filter params (name=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:14.996931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:14.998288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:15.003242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 503/1001] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:15.008685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:15.036443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:15.071827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 499/1002] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:15.095268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bbf8c-4cd9-4d43-b437-7f5d38399716 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:15.114711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc8fa81-e425-4b8c-bee8-0c0f3843e671 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:15.141936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:15.151970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:15.153305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:15.153590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:15.153870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:15.154568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] RBAC: Adding query filter params (type=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:15.154877 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:15.156538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:15.161460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 504/1003] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:15 2022] GET /identity/v3/services?type=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:15.195904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bbf8c-4cd9-4d43-b437-7f5d38399716 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:15.233814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:15.234636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:15.234923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:15.235200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:15.235895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:15.237486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:15.250538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:15.251637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 07 10:14:15.252617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:15.256092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 500/1004] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:15.257983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:15.365208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37b4b8-f3d1-49e2-8333-c0909ff0245f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:15.400129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:15.400805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:15.401085 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:15.401358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:15.402083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] RBAC: Adding query filter params (name=compute) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:15.402392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:15.403774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:15.413187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 505/1005] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:14:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:15.414436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:15.450983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:15.525173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53876ba-12f4-45ac-aca1-f5ff99d21696 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:15.559364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89993ec-5e88-4f08-9a28-c66b923e408c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:15.560050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89993ec-5e88-4f08-9a28-c66b923e408c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:15.560335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89993ec-5e88-4f08-9a28-c66b923e408c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:15.560855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89993ec-5e88-4f08-9a28-c66b923e408c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:15.561554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89993ec-5e88-4f08-9a28-c66b923e408c None admin] RBAC: Adding query filter params (type=compute) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:15.561860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89993ec-5e88-4f08-9a28-c66b923e408c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:15.563208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89993ec-5e88-4f08-9a28-c66b923e408c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:15.571813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 501/1006] 10.222.0.16 () {56 vars in 1112 bytes} [Thu Jul 7 10:14:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:15.573903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:15.630640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:15.676895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9b67d-0991-4f44-a1cb-f79609bb96ad None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:15.712412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bac4b8-f927-42cd-9def-b4097859fd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:15.712412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac4b8-f927-42cd-9def-b4097859fd8b None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:15.712708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac4b8-f927-42cd-9def-b4097859fd8b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:15.712919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bac4b8-f927-42cd-9def-b4097859fd8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:15.713605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bac4b8-f927-42cd-9def-b4097859fd8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:15.715570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bac4b8-f927-42cd-9def-b4097859fd8b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:15.743863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 506/1007] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:14:15 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:14:16.293615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657813b-02ad-4ad0-af06-3a7267b3a8ff None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:16.294129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657813b-02ad-4ad0-af06-3a7267b3a8ff None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:16.294207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657813b-02ad-4ad0-af06-3a7267b3a8ff None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:16.295674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 502/1008] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:16.300852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a899-2ed4-47c5-8962-23473fb55b88 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:16.301148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a899-2ed4-47c5-8962-23473fb55b88 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:16.301429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd2a899-2ed4-47c5-8962-23473fb55b88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:16.318970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-abd2a899-2ed4-47c5-8962-23473fb55b88 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:16.320195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-abd2a899-2ed4-47c5-8962-23473fb55b88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:16.395912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 507/1009] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:16 2022] POST /identity/v3/auth/tokens => generated 1716 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:16.415320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657813b-02ad-4ad0-af06-3a7267b3a8ff None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:16.518309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:16.523561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657813b-02ad-4ad0-af06-3a7267b3a8ff None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:16.557788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:16.559137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:16.559420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:16.559691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:16.560373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:16.561990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:16.574134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:16.575105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 07 10:14:16.576382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:16.577455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 503/1010] 10.222.0.16 () {56 vars in 1126 bytes} [Thu Jul 7 10:14:16 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:16.581245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a899-2ed4-47c5-8962-23473fb55b88 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:16.617073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:16.683107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd2a899-2ed4-47c5-8962-23473fb55b88 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:16.717017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:16.717684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:16.717965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:16.718238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:16.718932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:16.719238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:16.720632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:16.730639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 508/1011] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:16 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:16.731912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:16.835657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834cb1ce-17d3-444c-a9bf-8efda1f34729 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:16.868823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4bb56-daf0-4d45-a211-471c29dac89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:16.869498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4bb56-daf0-4d45-a211-471c29dac89a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:16.869780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4bb56-daf0-4d45-a211-471c29dac89a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:16.870055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4bb56-daf0-4d45-a211-471c29dac89a None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:16.871277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4bb56-daf0-4d45-a211-471c29dac89a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:16.871583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4bb56-daf0-4d45-a211-471c29dac89a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:16.872953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4bb56-daf0-4d45-a211-471c29dac89a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:16.879867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 504/1012] 10.222.0.16 () {56 vars in 1122 bytes} [Thu Jul 7 10:14:16 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:16.883988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:16.988256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:16.992813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10701f3-2823-464f-8a92-12b68efd24c4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:17.028224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba20073-80a6-48e4-ab6c-8741f267e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:17.028873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba20073-80a6-48e4-ab6c-8741f267e25c None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:17.029152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba20073-80a6-48e4-ab6c-8741f267e25c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:17.029424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba20073-80a6-48e4-ab6c-8741f267e25c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:17.030150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba20073-80a6-48e4-ab6c-8741f267e25c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:17.031500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba20073-80a6-48e4-ab6c-8741f267e25c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:17.047913 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 509/1013] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:14:16 2022] POST /identity/v3/endpoints => generated 325 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:14:17.191444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed605-15f1-4945-a2f6-1adc503effb5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:17.191634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed605-15f1-4945-a2f6-1adc503effb5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:17.191955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032ed605-15f1-4945-a2f6-1adc503effb5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:17.193612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 505/1014] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:17.198749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36aa5e-bc6d-4d05-8a7c-96fdd8ab14a8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:17.199574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36aa5e-bc6d-4d05-8a7c-96fdd8ab14a8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:17.199936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec36aa5e-bc6d-4d05-8a7c-96fdd8ab14a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:17.216983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ec36aa5e-bc6d-4d05-8a7c-96fdd8ab14a8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:17.218177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec36aa5e-bc6d-4d05-8a7c-96fdd8ab14a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:17.293654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 510/1015] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:17 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:17.317052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ed605-15f1-4945-a2f6-1adc503effb5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:17.345773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:17.422118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032ed605-15f1-4945-a2f6-1adc503effb5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:17.456129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98390d96-246d-4cc6-8c17-a23489aeaa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:17.456726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98390d96-246d-4cc6-8c17-a23489aeaa96 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:17.457014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98390d96-246d-4cc6-8c17-a23489aeaa96 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:17.457295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98390d96-246d-4cc6-8c17-a23489aeaa96 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:17.458031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98390d96-246d-4cc6-8c17-a23489aeaa96 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:17.459392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98390d96-246d-4cc6-8c17-a23489aeaa96 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:17.473345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 506/1016] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:14:17 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:17.848688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e845c6-9af7-48be-81fe-e984c9f2137f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:17.849241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e845c6-9af7-48be-81fe-e984c9f2137f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:17.849241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e845c6-9af7-48be-81fe-e984c9f2137f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:17.850740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 511/1017] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:17.856346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551b04a-086a-4e35-9518-3f04703e5a69 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:17.856650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551b04a-086a-4e35-9518-3f04703e5a69 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:17.856926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551b04a-086a-4e35-9518-3f04703e5a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:17.873687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3551b04a-086a-4e35-9518-3f04703e5a69 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:17.874863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3551b04a-086a-4e35-9518-3f04703e5a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:17.951872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 507/1018] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:17.973603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e845c6-9af7-48be-81fe-e984c9f2137f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:18.060248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:18.079404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e845c6-9af7-48be-81fe-e984c9f2137f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:18.113338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:18.114074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:18.114354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:18.114627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:18.125562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:18.134013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:18.135009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:18.136272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:18.137407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 512/1019] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jul 7 10:14:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:18.141491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551b04a-086a-4e35-9518-3f04703e5a69 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:18.155880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:18.244972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551b04a-086a-4e35-9518-3f04703e5a69 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:18.278921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:18.279593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:18.279910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:18.280188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:18.280931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:18.281240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:18.282592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:18.295571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 508/1020] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jul 7 10:14:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:18.296803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:18.395631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cdf55a-bd36-48e1-98a0-cb2eaf61f76d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:18.429398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:18.430345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:18.430776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:18.431223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:18.442267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:18.443934 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:18.450320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:18.462247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:18.463155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:18.468635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:18.468635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:18.468635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:18.468635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:18.468635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 513/1021] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:18.471266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:18.551083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:18.577300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e30d4-af05-4457-8a79-e4c76751ed9d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:18.611982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:18.612670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:18.612962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:18.613242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:18.613989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:18.614304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:18.615718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:18.629548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 509/1022] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:18.634844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:18.735938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095417d9-818d-4e8e-ad4e-c9363c48f670 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:18.769387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:18.770174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:18.770458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:18.770733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:18.778806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:18.780284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:14:18.794180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:18.795164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:14:18.796680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:18.801662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 514/1023] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:14:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:18.801801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:18.903961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2751e1-c282-4d79-bed4-d7b90e7073cb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:18.937245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd33aca-cff0-4954-8567-db736a6b136b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:18.938117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd33aca-cff0-4954-8567-db736a6b136b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:18.938573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd33aca-cff0-4954-8567-db736a6b136b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:18.939012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd33aca-cff0-4954-8567-db736a6b136b None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:18.940014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd33aca-cff0-4954-8567-db736a6b136b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:18.940527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd33aca-cff0-4954-8567-db736a6b136b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:18.942147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd33aca-cff0-4954-8567-db736a6b136b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:18.943962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:18.962751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 510/1024] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:14:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:18.969237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.030063 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:19.079589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede69ba8-29ec-47eb-b720-0614def52549 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.099590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53376d7-9224-4531-8b30-68f67c5f2f6a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.100391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53376d7-9224-4531-8b30-68f67c5f2f6a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.100709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53376d7-9224-4531-8b30-68f67c5f2f6a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.102231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 511/1025] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:19.107140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fe434-6bdc-4a2b-8649-a78ec2472a60 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.107433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fe434-6bdc-4a2b-8649-a78ec2472a60 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.107713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8fe434-6bdc-4a2b-8649-a78ec2472a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.114009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:19.114656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.114934 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.115205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.124869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1d8fe434-6bdc-4a2b-8649-a78ec2472a60 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:19.126112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1d8fe434-6bdc-4a2b-8649-a78ec2472a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:19.186911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:19.188393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:19.195559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 512/1026] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:19.218878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fe434-6bdc-4a2b-8649-a78ec2472a60 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.248382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:19.296308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 515/1027] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:18 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:19.301689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.325071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8fe434-6bdc-4a2b-8649-a78ec2472a60 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.350991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:19.363382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:19.364215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.364503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.364784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.365957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:19.367380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:19.374140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 513/1028] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:19.375019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.408657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0ace9-18eb-4248-9424-1e44a1477a96 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.446593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:19.447391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.447680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.447995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.458915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:19.460414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:19.477959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:19.478890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:19.480560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:19.480560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:19.480560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:19.480560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:19.480813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd28fa-bf92-4059-9b40-8ca499f71568 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.484259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 516/1029] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:19.485525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.499042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:19.521180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a451845-795c-4a68-99ac-73e9fbb0869d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:19.521850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a451845-795c-4a68-99ac-73e9fbb0869d None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.522128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a451845-795c-4a68-99ac-73e9fbb0869d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.522402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a451845-795c-4a68-99ac-73e9fbb0869d None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.523123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a451845-795c-4a68-99ac-73e9fbb0869d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:19.524570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a451845-795c-4a68-99ac-73e9fbb0869d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:19.553515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 514/1030] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:14:19 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:14:19.567273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80d490-1860-49e0-a03f-7b5ed03c766a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.567597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80d490-1860-49e0-a03f-7b5ed03c766a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.567873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e80d490-1860-49e0-a03f-7b5ed03c766a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.569437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 515/1031] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:19.575733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3aa9-7a6b-445c-85fe-d7f3c3945e6c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.576063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3aa9-7a6b-445c-85fe-d7f3c3945e6c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.576334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05c3aa9-7a6b-445c-85fe-d7f3c3945e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.593861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d05c3aa9-7a6b-445c-85fe-d7f3c3945e6c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:19.595097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d05c3aa9-7a6b-445c-85fe-d7f3c3945e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:19.614444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30d0e30-e20b-4e69-a8dd-cecc330bee62 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.637195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:19.653839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:19.654659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.655007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.655357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.656193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:19.656572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:19.657962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:19.671511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 517/1032] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:19.677021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.683776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 516/1033] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:19.707410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3aa9-7a6b-445c-85fe-d7f3c3945e6c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.756040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:19.789254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a127e-afae-4bb8-9c89-ccf15c353e19 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.820884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05c3aa9-7a6b-445c-85fe-d7f3c3945e6c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.825884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:19.826690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.826979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.827258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.834679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:19.836339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:14:19.852178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:19.853125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:19.854028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 518/1034] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:19.857160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:19.858202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:19.858306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:19.858501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] PATH_INFO: `/v3/services/image` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:19.859014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.859175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:19.860542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:19.873117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:19.874123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:19.875044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 517/1035] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jul 7 10:14:19 2022] GET /identity/v3/services/image => generated 86 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:19.880102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:19.916886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:19.967237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f098d9c3-97dd-4f7d-8193-890185c46ce3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.988475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7aca5f-5956-46b3-a3c9-cb01ef480934 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:19.998756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:20.005555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:20.006213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:20.006491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:20.006761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:20.007531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:20.007864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:20.009325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:20.021865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 519/1036] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:14:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:20.027566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:20.027708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:20.028523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:20.028987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:20.029416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:20.030275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] RBAC: Adding query filter params (name=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:20.030733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:20.032275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:20.037612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 518/1037] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:20.043108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:20.136711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e8e5-25b7-4340-b268-d8412e165f76 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:20.142810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:20.146162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2100983-13fe-42d2-8fc1-a481075e6f32 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:20.178050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:20.178702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:20.178985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:20.179261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:20.180116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] RBAC: Adding query filter params (role.id=0504664beb9b4fd6a50b4b64828b071f, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=0914dada5c70437ba49688789eb31ff9) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:20.180428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:20.181847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:20.181970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:20.182507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:20.182790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:20.183093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:20.183818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] RBAC: Adding query filter params (type=image) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:20.184126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:20.185727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:20.187807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 520/1038] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:20 2022] GET /identity/v3/role_assignments?user.id=0914dada5c70437ba49688789eb31ff9&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=0504664beb9b4fd6a50b4b64828b071f => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:20.199852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 519/1039] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:20.205651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:20.268392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:20.318767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d02d2a-c7e1-4064-9eda-1968354cf029 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:20.357106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1978d35-469f-45b2-9802-b2579c2dfc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:20.357769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1978d35-469f-45b2-9802-b2579c2dfc29 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:20.358047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1978d35-469f-45b2-9802-b2579c2dfc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:20.358318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1978d35-469f-45b2-9802-b2579c2dfc29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:20.359101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1978d35-469f-45b2-9802-b2579c2dfc29 None admin] RBAC: Adding query filter params (interface=public, service_id=00bfcbf03b064e88ba7847566008dd16, region_id=RegionOne) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:20.359412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1978d35-469f-45b2-9802-b2579c2dfc29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:20.360801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1978d35-469f-45b2-9802-b2579c2dfc29 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:20.366158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 521/1040] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:14:20 2022] GET /identity/v3/endpoints?service_id=00bfcbf03b064e88ba7847566008dd16&interface=public®ion_id=RegionOne => generated 194 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:20.372491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:20.476976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac22d7d-d5ee-4c2f-8a52-e218cacc394c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:20.514890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefa18e6-19c5-4189-b047-9fa95ffd9718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:20.514890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa18e6-19c5-4189-b047-9fa95ffd9718 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:20.514890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa18e6-19c5-4189-b047-9fa95ffd9718 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:20.514890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefa18e6-19c5-4189-b047-9fa95ffd9718 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:20.515701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa18e6-19c5-4189-b047-9fa95ffd9718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:20.515799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefa18e6-19c5-4189-b047-9fa95ffd9718 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:20.524031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 520/1041] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:14:20 2022] GET /identity/v3/services => generated 1957 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:14:21.659010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ab50-e745-4a0a-8b4b-7729164a9f57 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:21.659519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ab50-e745-4a0a-8b4b-7729164a9f57 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:21.659627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c731ab50-e745-4a0a-8b4b-7729164a9f57 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:21.661128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 522/1042] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:21.666107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13048a-d5e1-405f-852d-d9701fa19b02 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:21.666498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13048a-d5e1-405f-852d-d9701fa19b02 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:21.666666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa13048a-d5e1-405f-852d-d9701fa19b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:21.684077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fa13048a-d5e1-405f-852d-d9701fa19b02 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:21.685276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fa13048a-d5e1-405f-852d-d9701fa19b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:21.708965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:21.754914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 521/1043] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:21 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:21.780273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731ab50-e745-4a0a-8b4b-7729164a9f57 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:21.885437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c731ab50-e745-4a0a-8b4b-7729164a9f57 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:21.903152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:21.921282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b1e0df-3d83-4ea1-a8de-93144e3009db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:21.922075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e0df-3d83-4ea1-a8de-93144e3009db None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:21.922359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e0df-3d83-4ea1-a8de-93144e3009db None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:21.922641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b1e0df-3d83-4ea1-a8de-93144e3009db None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:21.923848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e0df-3d83-4ea1-a8de-93144e3009db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:21.925289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b1e0df-3d83-4ea1-a8de-93144e3009db None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:21.931512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 523/1044] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:21.933035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13048a-d5e1-405f-852d-d9701fa19b02 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.035223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa13048a-d5e1-405f-852d-d9701fa19b02 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:22.068939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c390b7-c49b-4836-a248-38da7947cdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:22.069622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c390b7-c49b-4836-a248-38da7947cdb4 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.069906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c390b7-c49b-4836-a248-38da7947cdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.070183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c390b7-c49b-4836-a248-38da7947cdb4 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.070896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c390b7-c49b-4836-a248-38da7947cdb4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:22.072356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c390b7-c49b-4836-a248-38da7947cdb4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:22.099780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 522/1045] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:14:21 2022] POST /identity/v3/users => generated 285 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:22.186211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290290d-bf36-4a92-9489-67f688ef7254 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.186495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290290d-bf36-4a92-9489-67f688ef7254 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.186771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290290d-bf36-4a92-9489-67f688ef7254 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.188349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 524/1046] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:22.193737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef7845-8b06-49d4-9ec2-a02e86835244 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.194033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef7845-8b06-49d4-9ec2-a02e86835244 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.194312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fef7845-8b06-49d4-9ec2-a02e86835244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.212252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1fef7845-8b06-49d4-9ec2-a02e86835244 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:22.213464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1fef7845-8b06-49d4-9ec2-a02e86835244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:22.224845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:22.289197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 523/1047] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:22.311701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290290d-bf36-4a92-9489-67f688ef7254 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.414840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290290d-bf36-4a92-9489-67f688ef7254 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:22.449192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a46032-a93f-43c5-a5f7-2cc348086bc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:22.450003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a46032-a93f-43c5-a5f7-2cc348086bc9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.450279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a46032-a93f-43c5-a5f7-2cc348086bc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.450548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a46032-a93f-43c5-a5f7-2cc348086bc9 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.461130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a46032-a93f-43c5-a5f7-2cc348086bc9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c3a46032-a93f-43c5-a5f7-2cc348086bc9 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:22.469173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:22.470139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:22.471021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 525/1048] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jul 7 10:14:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:22.476590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef7845-8b06-49d4-9ec2-a02e86835244 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.540695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa3dc7-da2b-4630-8d1e-251e5c87f7a3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.540868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa3dc7-da2b-4630-8d1e-251e5c87f7a3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.541114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aa3dc7-da2b-4630-8d1e-251e5c87f7a3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.543807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 526/1049] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:22.548560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76381f02-433d-4954-8595-391e321ed9ad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.548913 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76381f02-433d-4954-8595-391e321ed9ad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.549119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76381f02-433d-4954-8595-391e321ed9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.566814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-76381f02-433d-4954-8595-391e321ed9ad None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:22.568126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-76381f02-433d-4954-8595-391e321ed9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:22.588728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fef7845-8b06-49d4-9ec2-a02e86835244 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:22.613755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:22.626070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:22.626742 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.627023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.627302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.628077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:22.628384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:22.629749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:22.638035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 524/1050] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jul 7 10:14:22 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:22.643368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.648251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 527/1051] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:22.651400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:22.671464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76381f02-433d-4954-8595-391e321ed9ad None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.747206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbffd4b5-0139-4618-8384-a91e73c36584 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:22.778542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76381f02-433d-4954-8595-391e321ed9ad None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:22.780768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:22.781656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.781845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.782111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.793551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:22.794927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:22.815499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:22.816453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:22.817324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:22.817324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:22.817324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:22.817324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:22.817324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 525/1052] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:22.818763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:22.819549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.819857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.820133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] PATH_INFO: `/v3/services/image` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.820779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:22.822212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:22.822350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:22.833861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:22.834893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:22.836606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 528/1053] 10.222.0.16 () {56 vars in 1105 bytes} [Thu Jul 7 10:14:22 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:22.841593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.882614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:22.891197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:22.930163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f051f394-3fe9-46cc-8834-e8fc4fd57bba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:22.953976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9762e53c-16d5-4a0b-a92b-ca954110fd95 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:22.963462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:22.964682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.964990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.965258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.965984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:22.966294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:22.967664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:22.979353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 526/1054] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:22.984921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:22.996297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:22.997270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:22.997711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:22.997986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:22.998678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] RBAC: Adding query filter params (name=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:22.998987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.000368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.010086 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 529/1055] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:23.011195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:23.088307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d3b529-13bc-4e51-ac6a-095c4dccecde None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:23.107244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:23.116791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bfbd8b-1078-4b28-b1ab-4bfd33b62d87 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:23.120226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:23.126941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:23.128032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:23.128316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:23.128592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:23.136543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.137986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:14:23.153265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:23.154207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:14:23.155134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:23.156504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:23.157224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:23.157512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:23.157791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:23.158489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:23.158801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.159258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 527/1056] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:14:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:23.159946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:23.160536 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.166127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 530/1057] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:23.171916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:23.272600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138d571f-5b43-486b-ad18-48ae49626e9b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:23.282812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c731cdf-f70a-47bb-b75c-05e832112bc0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:23.309221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:23.309905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:23.310188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:23.310730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:23.311567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:23.312061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.313375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.319234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:23.319931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:23.320218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:23.320494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:23.321233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.323010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.332645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 528/1058] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:14:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:23.338565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:23.351780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 531/1059] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:14:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:14:23.388520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:23.463242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60896264-e829-4446-9d9c-d86621c383f2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:23.510170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:23.511515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:23.511821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:23.512096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:23.522936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.524515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:23.542958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:23.543930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:23.544867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:23.544867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:23.544867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:23.544867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:23.544867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 529/1060] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:23.549728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:23.577594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:23.656882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97f385b-587e-4564-b812-4962c0105094 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:23.690422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:23.691153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:23.691369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:23.691646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:23.692394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:23.692704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.694075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.706887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 532/1061] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:23.712152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:23.796898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:23.814937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d36b0bf-29a0-4656-ab4a-f3a97a082f29 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:23.851213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:23.852050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:23.852336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:23.852608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:23.859284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:23.860737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:14:23.874707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:23.875613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:23.876554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 530/1062] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:23.882753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:23.969857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:23.987005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01c421-58e7-47fc-8cd4-e85133fd0042 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:24.023948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bcca3d-b73c-4538-a611-e3600dec9e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:24.024611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcca3d-b73c-4538-a611-e3600dec9e86 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.024895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcca3d-b73c-4538-a611-e3600dec9e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.025170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bcca3d-b73c-4538-a611-e3600dec9e86 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.025944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcca3d-b73c-4538-a611-e3600dec9e86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:24.026254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcca3d-b73c-4538-a611-e3600dec9e86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:24.028340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bcca3d-b73c-4538-a611-e3600dec9e86 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:24.045096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 533/1063] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:14:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:24.046623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:24.149744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621b55f0-123d-4ea1-a893-897c07c67208 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:24.183448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:24.184366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.184650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.184928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/0914dada5c70437ba49688789eb31ff9/roles/0504664beb9b4fd6a50b4b64828b071f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.187400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=0914dada5c70437ba49688789eb31ff9, role_id=0504664beb9b4fd6a50b4b64828b071f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:24.188867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:24.200046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca7dcf-1f1d-4a79-80c3-d300966419de None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.200334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca7dcf-1f1d-4a79-80c3-d300966419de None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.200608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ca7dcf-1f1d-4a79-80c3-d300966419de None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.202151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 534/1064] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:14:24.203940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 531/1065] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:14:24 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/0914dada5c70437ba49688789eb31ff9/roles/0504664beb9b4fd6a50b4b64828b071f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:14:24.207163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a7fa2-125e-454c-b4ef-91e1e9907daa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.207456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a7fa2-125e-454c-b4ef-91e1e9907daa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.207731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a7fa2-125e-454c-b4ef-91e1e9907daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.226052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-705a7fa2-125e-454c-b4ef-91e1e9907daa None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:24.227180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-705a7fa2-125e-454c-b4ef-91e1e9907daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:24.300850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 535/1066] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:24.324661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:24.328637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:24.432429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7915136-aca7-4e12-aabb-457acc9724b5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:24.466214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:24.467013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.467293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.467569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.477500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:24.485301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:24.486287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:24.487586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 532/1067] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:14:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:24.493051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a7fa2-125e-454c-b4ef-91e1e9907daa None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:24.575443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:24.601033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705a7fa2-125e-454c-b4ef-91e1e9907daa None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:24.634481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:24.635152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.635429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.635770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.637001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:24.637307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:24.638660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:24.651506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 536/1068] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:14:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:24.653090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:24.717702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:24.757717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9555a9d-d6ed-4860-bc08-3fb47c4ef3fc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:24.791049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:24.791980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.792270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.792546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] PATH_INFO: `/v3/users/b9eebcdc7e7945ab948f54731774dd97` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.806138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] RBAC: Authorizing `identity:get_user(user_id=b9eebcdc7e7945ab948f54731774dd97)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:24.807597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:24.814232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 533/1069] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:24 2022] GET /identity/v3/users/b9eebcdc7e7945ab948f54731774dd97 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:24.815507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:24.916406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516d0069-5f4f-4236-bdd2-f1292f253462 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:24.949870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:24.950518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:24.950796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:24.951069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:24.954530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:25.018160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:25.019516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:25.122272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 537/1070] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 10:14:25.124101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:25.225249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0454303-83be-4fb7-b0f1-2d680723a622 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:25.248866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:25.262049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1c426-7382-404c-8a8f-41bb560091cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:25.262918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c426-7382-404c-8a8f-41bb560091cc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:25.263203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c426-7382-404c-8a8f-41bb560091cc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:25.263479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1c426-7382-404c-8a8f-41bb560091cc None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:25.264712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1c426-7382-404c-8a8f-41bb560091cc None admin] RBAC: Authorizing `identity:get_project(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:25.266175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1c426-7382-404c-8a8f-41bb560091cc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:25.268053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 534/1071] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:25 2022] GET /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:25.274087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:25.311479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:25.380042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1567442d-3e9f-45bc-98e0-9fe86609d257 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:25.415298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:25.416009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:25.416255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:25.416525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:25.417369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1, user.id=b9eebcdc7e7945ab948f54731774dd97) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:25.417549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c90ea-e237-4b77-a6fc-89aa913b3933 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:25.417666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:25.417842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c90ea-e237-4b77-a6fc-89aa913b3933 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:25.418123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c90ea-e237-4b77-a6fc-89aa913b3933 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:25.419116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:25.419649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 535/1072] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:25.424217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a6204-7b49-479d-9e94-547b90f3bf08 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:25.424556 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a6204-7b49-479d-9e94-547b90f3bf08 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:25.432196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15a6204-7b49-479d-9e94-547b90f3bf08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:25.432385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 538/1073] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:25 2022] GET /identity/v3/role_assignments?user.id=b9eebcdc7e7945ab948f54731774dd97&scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:25.449865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f15a6204-7b49-479d-9e94-547b90f3bf08 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:25.451134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f15a6204-7b49-479d-9e94-547b90f3bf08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:25.532902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 536/1074] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:25.555482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:25.656432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:25.660150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d548f97a-7a8b-4113-8e32-0ce47b367236 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:25.695727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:25.696409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:25.696693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:25.696972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:25.765284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:25.766739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:25.862374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:25.867548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 539/1075] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:25.873416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a6204-7b49-479d-9e94-547b90f3bf08 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:25.972235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15a6204-7b49-479d-9e94-547b90f3bf08 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:25.991951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:26.007237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ffe46b-19f5-4b42-b4d1-f61820422817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:26.008069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe46b-19f5-4b42-b4d1-f61820422817 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.008357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe46b-19f5-4b42-b4d1-f61820422817 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.008636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ffe46b-19f5-4b42-b4d1-f61820422817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.021305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffe46b-19f5-4b42-b4d1-f61820422817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:26.022755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ffe46b-19f5-4b42-b4d1-f61820422817 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-84ffe46b-19f5-4b42-b4d1-f61820422817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:26.040247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:26.041148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:26.042071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:26.042071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:26.042071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:26.042071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:26.044069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 537/1076] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:26.047353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:26.148077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e57007-2526-4597-b756-6a610b09f5fb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:26.183024 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b151195-4627-46b4-9ede-061983e63be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:26.183688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b151195-4627-46b4-9ede-061983e63be5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.184000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b151195-4627-46b4-9ede-061983e63be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.184277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b151195-4627-46b4-9ede-061983e63be5 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.184990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b151195-4627-46b4-9ede-061983e63be5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:26.185561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b151195-4627-46b4-9ede-061983e63be5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:26.186933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b151195-4627-46b4-9ede-061983e63be5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:26.199282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 540/1077] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:26.237015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1df2cc-493b-439c-9eab-df2a461ec9ef None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.237331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1df2cc-493b-439c-9eab-df2a461ec9ef None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.237585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1df2cc-493b-439c-9eab-df2a461ec9ef None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.239151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 538/1078] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:26.245353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b5a8d-b6f9-4eed-9fc6-96e479cea61b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.245663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b5a8d-b6f9-4eed-9fc6-96e479cea61b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.245916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9b5a8d-b6f9-4eed-9fc6-96e479cea61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.264370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3c9b5a8d-b6f9-4eed-9fc6-96e479cea61b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:26.265602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3c9b5a8d-b6f9-4eed-9fc6-96e479cea61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:26.326108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:26.339992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 541/1079] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:26.363062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1df2cc-493b-439c-9eab-df2a461ec9ef None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:26.468895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1df2cc-493b-439c-9eab-df2a461ec9ef None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:26.507046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:26.507927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.508212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.508487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.520687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:26.528932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:26.529931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:26.530797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 539/1080] 10.222.0.16 () {56 vars in 1120 bytes} [Thu Jul 7 10:14:26 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:26.536179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b5a8d-b6f9-4eed-9fc6-96e479cea61b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:26.631836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9b5a8d-b6f9-4eed-9fc6-96e479cea61b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:26.664204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:26.664858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.665137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.665411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.666135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:26.666440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:26.668383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:26.676014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 542/1081] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jul 7 10:14:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:26.681559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:26.712375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:26.778663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4be015-2ccf-45c2-ad2b-46fa6c3161fe None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:26.810899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:26.811680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.811989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.812264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.823209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:26.824689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:26.841617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:26.842515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:26.843583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:26.843583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:26.843583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:26.843583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:26.843583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 540/1082] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:26.848411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:26.899766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:26.946068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a1cfd-820d-43e5-89e2-93bbec30f481 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:26.978974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:26.979628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:26.979935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:26.980206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:26.980911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:26.981214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:26.982562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:26.993949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 543/1083] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:26.999550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.083322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:27.100378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4d18f-6836-420b-845f-a3334d256425 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.132920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.133718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.134007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.134303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.142381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:27.143841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:14:27.158101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:27.159020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:27.160008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 541/1084] 10.222.0.16 () {56 vars in 1093 bytes} [Thu Jul 7 10:14:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:27.164921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.262625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f260d74-b82b-4fa9-9955-9467b542d338 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.275328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:27.296728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e1da1-c1c3-451f-829e-720f979858e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.297398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e1da1-c1c3-451f-829e-720f979858e9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.297683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e1da1-c1c3-451f-829e-720f979858e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.297956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e1da1-c1c3-451f-829e-720f979858e9 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.298789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e1da1-c1c3-451f-829e-720f979858e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:27.299100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e1da1-c1c3-451f-829e-720f979858e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:27.300549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e1da1-c1c3-451f-829e-720f979858e9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:27.318079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 544/1085] 10.222.0.16 () {56 vars in 1133 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:27.323696 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.421869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30279bce-cad6-4adc-ac59-d79d1fb1ae68 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.432996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b5b3-3f67-4790-8ea5-25138e1be502 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.433292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b5b3-3f67-4790-8ea5-25138e1be502 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.433580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17b5b3-3f67-4790-8ea5-25138e1be502 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.435073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 545/1086] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:27.440143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b372d54-b354-4072-b5c7-32bfc0a305da None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.440432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b372d54-b354-4072-b5c7-32bfc0a305da None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.440712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b372d54-b354-4072-b5c7-32bfc0a305da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.454939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.455586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.455899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.456173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.457609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3b372d54-b354-4072-b5c7-32bfc0a305da None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:27.458809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3b372d54-b354-4072-b5c7-32bfc0a305da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:27.471501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:27.525517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:27.525988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 546/1087] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:27.527012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:27.547676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b372d54-b354-4072-b5c7-32bfc0a305da None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.623219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 542/1088] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 10:14:27.628932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.644333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:27.647474 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b372d54-b354-4072-b5c7-32bfc0a305da None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.671842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:27.681589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.682816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.683083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.683358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.693657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:27.701310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:27.702532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:27.703585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 547/1089] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:27.708412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.726374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de33e47-fa0a-41ae-a477-06004e1f1fc0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.760322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.761134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.761412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.761685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.771578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:27.773033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:27.790485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:27.791375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:27.793078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:14:27.793078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:27.793078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:14:27.793078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:27.793078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 543/1090] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:27.797717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.807509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f99de1-7aca-4f2a-b366-c58c73311cae None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.841552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.842217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.842501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.842778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.843518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:27.843857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:27.845232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:27.854596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 548/1091] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:27.860306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.882509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:27.890108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:27.896597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137ad7f-7a88-4491-9058-71be23308a36 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.929779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.930439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.930724 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.931000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.931783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:27.932055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:27.933441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:27.944618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 544/1092] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:27.950656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:27.957131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749bcf6c-6d46-4798-860e-69783f5b31d3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:27.992047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:27.992910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:27.993193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:27.993466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] PATH_INFO: `/v3/users/b9eebcdc7e7945ab948f54731774dd97` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:27.994652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] RBAC: Authorizing `identity:get_user(user_id=b9eebcdc7e7945ab948f54731774dd97)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:27.996112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:27.998013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 549/1093] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/users/b9eebcdc7e7945ab948f54731774dd97 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:28.003319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:28.052696 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefab53d-aed9-455f-9fbf-95c593dcb0e7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:28.085700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:28.086487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:28.086778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:28.087056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:28.093268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:28.094137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:28.095685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:28.103307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2d6e87-43ff-4554-aa98-f88481a30265 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:14:28.110893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:28.111833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:28.112845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 545/1094] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:14:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:14:28.117771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:28.135400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:28.139967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:28.140824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:28.141108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:28.141383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:28.142553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] RBAC: Authorizing `identity:get_project(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:28.144012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:28.145705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 550/1095] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:28 2022] GET /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:28.152284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:28.217838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b48c675-d6ec-43aa-a6f0-b72ad4ff10bc None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:28.248687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa73ec8-d639-468f-8fad-7935d11df822 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:28.252742 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:28.253408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:28.253694 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:28.253972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:28.254763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:28.255076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:28.256563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:28.267850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 546/1096] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:14:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:28.273715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:28.282181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55929c91-47f8-42de-aca6-3f9be73a4503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:28.283125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55929c91-47f8-42de-aca6-3f9be73a4503 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:28.283434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55929c91-47f8-42de-aca6-3f9be73a4503 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:28.283713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55929c91-47f8-42de-aca6-3f9be73a4503 None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1/users/b9eebcdc7e7945ab948f54731774dd97/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:28.285740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55929c91-47f8-42de-aca6-3f9be73a4503 None admin] RBAC: Authorizing `identity:create_grant(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1, user_id=b9eebcdc7e7945ab948f54731774dd97, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:28.287161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55929c91-47f8-42de-aca6-3f9be73a4503 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:28.299855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 551/1097] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:14:28 2022] PUT /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1/users/b9eebcdc7e7945ab948f54731774dd97/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:14:28.358964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:28.383732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9d3f52-0dc0-4031-bf4e-9920428da6d8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:28.418627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddc8d8a-bd8a-4236-a72d-e272201a7df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:28.419296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8d8a-bd8a-4236-a72d-e272201a7df2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:28.419585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8d8a-bd8a-4236-a72d-e272201a7df2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:28.419911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddc8d8a-bd8a-4236-a72d-e272201a7df2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:28.420757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc8d8a-bd8a-4236-a72d-e272201a7df2 None admin] RBAC: Adding query filter params (role.id=0504664beb9b4fd6a50b4b64828b071f, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=0914dada5c70437ba49688789eb31ff9) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:28.421075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc8d8a-bd8a-4236-a72d-e272201a7df2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:28.422521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddc8d8a-bd8a-4236-a72d-e272201a7df2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:28.429015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 547/1098] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:28 2022] GET /identity/v3/role_assignments?user.id=0914dada5c70437ba49688789eb31ff9&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=0504664beb9b4fd6a50b4b64828b071f => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:14:30.294433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb7ab-a940-4ed3-a020-3b12ad43e25b None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:30.294994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb7ab-a940-4ed3-a020-3b12ad43e25b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:30.294994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffb7ab-a940-4ed3-a020-3b12ad43e25b None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:30.296398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 552/1099] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:30.301626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196982d-4c0d-4dd7-a6e7-6a756c3dd451 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:30.302053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196982d-4c0d-4dd7-a6e7-6a756c3dd451 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:30.302186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b196982d-4c0d-4dd7-a6e7-6a756c3dd451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:30.318757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b196982d-4c0d-4dd7-a6e7-6a756c3dd451 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:30.319968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b196982d-4c0d-4dd7-a6e7-6a756c3dd451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:30.384823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 548/1100] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:30.408339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb7ab-a940-4ed3-a020-3b12ad43e25b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:30.426769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:30.508554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffb7ab-a940-4ed3-a020-3b12ad43e25b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:30.541987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:30.542772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:30.543053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:30.543326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:30.553147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:30.561486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:30.562635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:30.563670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 553/1101] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:14:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:30.568780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196982d-4c0d-4dd7-a6e7-6a756c3dd451 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:30.634423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:30.664805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b196982d-4c0d-4dd7-a6e7-6a756c3dd451 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:30.698894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:30.699554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:30.699866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:30.700145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:30.700887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:30.701194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:30.702564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:30.710328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 549/1102] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:14:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:30.715982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:30.790563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:30.812224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caef2f7-6b85-438e-9ecf-1074406014e1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:30.843401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:30.844282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:30.844561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:30.844831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] PATH_INFO: `/v3/users/b9eebcdc7e7945ab948f54731774dd97` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:30.845991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] RBAC: Authorizing `identity:get_user(user_id=b9eebcdc7e7945ab948f54731774dd97)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:30.847407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:30.849011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 554/1103] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:30 2022] GET /identity/v3/users/b9eebcdc7e7945ab948f54731774dd97 => generated 285 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:30.855038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:30.951876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c1426f-d5de-459e-b6de-f971220d9bc8 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:30.979896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:30.985030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:30.985669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:30.985956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:30.986232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:31.050428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:31.051897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:31.147662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 550/1104] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:31.153974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:31.250492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f583c1-e8ba-4c7e-9612-9193d87b474f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:31.282905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334fbb5b-5494-496d-80ba-081df60d51a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:31.283763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fbb5b-5494-496d-80ba-081df60d51a0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:31.284038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fbb5b-5494-496d-80ba-081df60d51a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:31.284303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334fbb5b-5494-496d-80ba-081df60d51a0 None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:31.285453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fbb5b-5494-496d-80ba-081df60d51a0 None admin] RBAC: Authorizing `identity:get_project(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:31.286865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334fbb5b-5494-496d-80ba-081df60d51a0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:31.288560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 555/1105] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:31 2022] GET /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:31.295193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:31.324831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:31.390598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9e081e-d2b5-48c4-8bff-d786a7f0d3ba None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:31.423626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2926d6d-e1d4-48d9-8fcd-8c244ab1528d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:31.424241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926d6d-e1d4-48d9-8fcd-8c244ab1528d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:31.424522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926d6d-e1d4-48d9-8fcd-8c244ab1528d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:31.424794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2926d6d-e1d4-48d9-8fcd-8c244ab1528d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:31.425648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2926d6d-e1d4-48d9-8fcd-8c244ab1528d None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1, user.id=b9eebcdc7e7945ab948f54731774dd97) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:31.425958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2926d6d-e1d4-48d9-8fcd-8c244ab1528d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:31.427366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2926d6d-e1d4-48d9-8fcd-8c244ab1528d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:31.433518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 551/1106] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:31 2022] GET /identity/v3/role_assignments?user.id=b9eebcdc7e7945ab948f54731774dd97&scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:33.438523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a45ac-8471-4256-86fb-4aa40acd61e3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:33.439056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a45ac-8471-4256-86fb-4aa40acd61e3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:33.439056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973a45ac-8471-4256-86fb-4aa40acd61e3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:33.440443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 556/1107] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:33.445434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50759870-3962-41e9-b352-bb307bbcd853 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:33.445837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50759870-3962-41e9-b352-bb307bbcd853 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:33.445997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50759870-3962-41e9-b352-bb307bbcd853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:33.462787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-50759870-3962-41e9-b352-bb307bbcd853 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:33.464008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-50759870-3962-41e9-b352-bb307bbcd853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:33.528791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 552/1108] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:33.552933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a45ac-8471-4256-86fb-4aa40acd61e3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:33.569453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:33.652484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973a45ac-8471-4256-86fb-4aa40acd61e3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:33.684491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96aace0f-0857-4375-ba38-a57743a226e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:33.685529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aace0f-0857-4375-ba38-a57743a226e5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:33.685805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aace0f-0857-4375-ba38-a57743a226e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:33.686076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96aace0f-0857-4375-ba38-a57743a226e5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:33.687234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aace0f-0857-4375-ba38-a57743a226e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:33.688730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96aace0f-0857-4375-ba38-a57743a226e5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:33.690266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 557/1109] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:33.695870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50759870-3962-41e9-b352-bb307bbcd853 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:33.721838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:33.790728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50759870-3962-41e9-b352-bb307bbcd853 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:33.824038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65439b1-ec4f-4652-af40-0cf631a9e917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:33.824701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65439b1-ec4f-4652-af40-0cf631a9e917 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:33.824992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65439b1-ec4f-4652-af40-0cf631a9e917 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:33.825271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65439b1-ec4f-4652-af40-0cf631a9e917 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:33.825984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65439b1-ec4f-4652-af40-0cf631a9e917 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:33.827415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65439b1-ec4f-4652-af40-0cf631a9e917 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:33.845594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 553/1110] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:14:33 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:35.879494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec04a5-1d1d-4774-9e1d-453e3d2aee87 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:35.880091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec04a5-1d1d-4774-9e1d-453e3d2aee87 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:35.880091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ec04a5-1d1d-4774-9e1d-453e3d2aee87 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:35.881428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 558/1111] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:35.886559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086859-963a-4e58-80cb-b1981176fca7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:35.887004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086859-963a-4e58-80cb-b1981176fca7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:35.887105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c086859-963a-4e58-80cb-b1981176fca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:35.904621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c086859-963a-4e58-80cb-b1981176fca7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:35.905795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c086859-963a-4e58-80cb-b1981176fca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:35.969755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 554/1112] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:35.992234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec04a5-1d1d-4774-9e1d-453e3d2aee87 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:36.071850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:36.091264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ec04a5-1d1d-4774-9e1d-453e3d2aee87 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:36.122781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:36.123614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:36.123922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:36.124194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] PATH_INFO: `/v3/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:36.125727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] RBAC: Authorizing `identity:get_role(role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:36.127074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:36.128438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 559/1113] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:35 2022] GET /identity/v3/roles/7673131020174664baa4d10a6f38ce2e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:36.134141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086859-963a-4e58-80cb-b1981176fca7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:36.148210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:36.230963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c086859-963a-4e58-80cb-b1981176fca7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:36.263419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:36.264318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:36.264605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:36.264883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] PATH_INFO: `/v3/users/5854cb7ded52414b8050eb0806787f2f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:36.277385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] RBAC: Authorizing `identity:get_user(user_id=5854cb7ded52414b8050eb0806787f2f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:36.278820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:36.280477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 555/1114] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:36 2022] GET /identity/v3/users/5854cb7ded52414b8050eb0806787f2f => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:36.286201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:36.383596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52820d8-1b3d-4175-b037-6900316b135e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:36.416121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:36.416677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:36.416964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:36.417244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:36.435856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:36.483197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:36.484665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:36.581098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 560/1115] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:36.586886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:36.653518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:36.683610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7764a66-b157-4984-9eee-c126186ac811 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:36.716289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff1fe2-6591-4f11-9066-9eaa6d9e0748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:36.717137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff1fe2-6591-4f11-9066-9eaa6d9e0748 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:36.717421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff1fe2-6591-4f11-9066-9eaa6d9e0748 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:36.717699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff1fe2-6591-4f11-9066-9eaa6d9e0748 None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:36.718862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff1fe2-6591-4f11-9066-9eaa6d9e0748 None admin] RBAC: Authorizing `identity:get_project(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:36.720321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff1fe2-6591-4f11-9066-9eaa6d9e0748 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:36.721986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 556/1116] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:36 2022] GET /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:36.727852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:36.775795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:36.824408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22337104-3266-475c-88cd-be0e2558aa27 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:36.856797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900fdd1f-c72a-44a0-a93c-0ac828b2b75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:36.857437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fdd1f-c72a-44a0-a93c-0ac828b2b75d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:36.857717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fdd1f-c72a-44a0-a93c-0ac828b2b75d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:36.857992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900fdd1f-c72a-44a0-a93c-0ac828b2b75d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:36.858811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fdd1f-c72a-44a0-a93c-0ac828b2b75d None admin] RBAC: Adding query filter params (role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1, user.id=5854cb7ded52414b8050eb0806787f2f) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:36.859122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fdd1f-c72a-44a0-a93c-0ac828b2b75d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:36.860554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900fdd1f-c72a-44a0-a93c-0ac828b2b75d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:36.865768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 561/1117] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:36 2022] GET /identity/v3/role_assignments?user.id=5854cb7ded52414b8050eb0806787f2f&scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1&role.id=7673131020174664baa4d10a6f38ce2e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:38.840631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23197eb-e333-4f71-9f8e-9419e2a32719 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:38.841162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23197eb-e333-4f71-9f8e-9419e2a32719 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:38.841265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23197eb-e333-4f71-9f8e-9419e2a32719 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:38.842555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 557/1118] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:38.847619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02688d11-3bfe-4c50-a758-e1805442bd1e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:38.848083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02688d11-3bfe-4c50-a758-e1805442bd1e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:38.848192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02688d11-3bfe-4c50-a758-e1805442bd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:38.865660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-02688d11-3bfe-4c50-a758-e1805442bd1e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:38.866845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-02688d11-3bfe-4c50-a758-e1805442bd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:38.930814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 562/1119] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:38.954051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23197eb-e333-4f71-9f8e-9419e2a32719 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:39.051675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23197eb-e333-4f71-9f8e-9419e2a32719 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:39.084215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:39.085062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:39.085352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:39.085623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] PATH_INFO: `/v3/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:39.087080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] RBAC: Authorizing `identity:get_role(role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:39.089003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:39.090332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 558/1120] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:38 2022] GET /identity/v3/roles/7673131020174664baa4d10a6f38ce2e => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:39.095923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02688d11-3bfe-4c50-a758-e1805442bd1e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:39.139916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:39.191721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02688d11-3bfe-4c50-a758-e1805442bd1e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:39.224700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:39.225559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:39.225836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:39.226107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] PATH_INFO: `/v3/users/5854cb7ded52414b8050eb0806787f2f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:39.227275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] RBAC: Authorizing `identity:get_user(user_id=5854cb7ded52414b8050eb0806787f2f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:39.228767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:39.230371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 563/1121] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:39 2022] GET /identity/v3/users/5854cb7ded52414b8050eb0806787f2f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:39.236227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:39.244806 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:39.333138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b24c57-3721-4e17-90e0-026342d262cf None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:39.365767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b9196b-8dd9-4966-b790-04e3417a3dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:39.366562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9196b-8dd9-4966-b790-04e3417a3dae None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:39.366846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9196b-8dd9-4966-b790-04e3417a3dae None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:39.367124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b9196b-8dd9-4966-b790-04e3417a3dae None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:39.368316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9196b-8dd9-4966-b790-04e3417a3dae None admin] RBAC: Authorizing `identity:get_project(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:39.370261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b9196b-8dd9-4966-b790-04e3417a3dae None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:39.372001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 559/1122] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:39 2022] GET /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:39.377809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:39.473304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4435fbb-9dff-4ef3-bbea-45ec5c0da546 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:39.485903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:39.506489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f36f148-928f-42c6-89c2-7d090f69dd3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:39.507367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36f148-928f-42c6-89c2-7d090f69dd3c None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:39.507644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36f148-928f-42c6-89c2-7d090f69dd3c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:39.507953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f36f148-928f-42c6-89c2-7d090f69dd3c None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1/users/5854cb7ded52414b8050eb0806787f2f/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:39.510700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36f148-928f-42c6-89c2-7d090f69dd3c None admin] RBAC: Authorizing `identity:create_grant(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1, user_id=5854cb7ded52414b8050eb0806787f2f, role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:39.512153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f36f148-928f-42c6-89c2-7d090f69dd3c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:39.525639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 564/1123] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:14:39 2022] PUT /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1/users/5854cb7ded52414b8050eb0806787f2f/roles/7673131020174664baa4d10a6f38ce2e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:14:41.476121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff2334-7992-4e5d-b5e0-5dfe124c45d9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:41.480294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff2334-7992-4e5d-b5e0-5dfe124c45d9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:41.480294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff2334-7992-4e5d-b5e0-5dfe124c45d9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:41.480294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 560/1124] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:41.483914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b4e3-aae0-4c45-a87a-a67477a9ed22 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:41.484327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b4e3-aae0-4c45-a87a-a67477a9ed22 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:41.484470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff97b4e3-aae0-4c45-a87a-a67477a9ed22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:41.501256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ff97b4e3-aae0-4c45-a87a-a67477a9ed22 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:41.502448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff97b4e3-aae0-4c45-a87a-a67477a9ed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:41.567287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 565/1125] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:41.590161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff2334-7992-4e5d-b5e0-5dfe124c45d9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:41.662035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:41.687278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff2334-7992-4e5d-b5e0-5dfe124c45d9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:41.719979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:41.720815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:41.721095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:41.721364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] PATH_INFO: `/v3/roles/7673131020174664baa4d10a6f38ce2e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:41.722809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] RBAC: Authorizing `identity:get_role(role_id=7673131020174664baa4d10a6f38ce2e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:41.724190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:41.725509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 561/1126] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:41 2022] GET /identity/v3/roles/7673131020174664baa4d10a6f38ce2e => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:41.731726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97b4e3-aae0-4c45-a87a-a67477a9ed22 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:41.826589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff97b4e3-aae0-4c45-a87a-a67477a9ed22 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:41.828113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:41.860539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:41.861396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:41.861672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:41.861938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] PATH_INFO: `/v3/users/5854cb7ded52414b8050eb0806787f2f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:41.863105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] RBAC: Authorizing `identity:get_user(user_id=5854cb7ded52414b8050eb0806787f2f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:41.864550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:41.866317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 566/1127] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:41 2022] GET /identity/v3/users/5854cb7ded52414b8050eb0806787f2f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:41.872311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:41.965685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395c0aa-0f6e-4916-9df3-c06d2320ef19 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:41.998160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:41.998733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:41.999015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:41.999288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:42.010373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:42.063629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:42.065081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:42.157162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 562/1128] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:42.162979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:42.259062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc6f995-5936-4ece-9b43-f063cfe065e9 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:42.291505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2755b7-a6ff-40c2-bc84-dfa544a372bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:42.292369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2755b7-a6ff-40c2-bc84-dfa544a372bd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:42.292653 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2755b7-a6ff-40c2-bc84-dfa544a372bd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:42.292929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2755b7-a6ff-40c2-bc84-dfa544a372bd None admin] PATH_INFO: `/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:42.294093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2755b7-a6ff-40c2-bc84-dfa544a372bd None admin] RBAC: Authorizing `identity:get_project(project_id=3058bf1ad7d74bcf9bff9120acb7f1c1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:42.295511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2755b7-a6ff-40c2-bc84-dfa544a372bd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:42.297196 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 567/1129] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:42 2022] GET /identity/v3/projects/3058bf1ad7d74bcf9bff9120acb7f1c1 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:42.303020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:42.345304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:42.399362 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd3f50-b00e-4196-816e-bbeba57b313b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:42.432738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fb377c-7d41-4504-8988-787843a6d6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:42.433382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb377c-7d41-4504-8988-787843a6d6f1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:42.433668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb377c-7d41-4504-8988-787843a6d6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:42.433949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fb377c-7d41-4504-8988-787843a6d6f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:42.435029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb377c-7d41-4504-8988-787843a6d6f1 None admin] RBAC: Adding query filter params (role.id=7673131020174664baa4d10a6f38ce2e, scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1, user.id=5854cb7ded52414b8050eb0806787f2f) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:42.435346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb377c-7d41-4504-8988-787843a6d6f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:42.436789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fb377c-7d41-4504-8988-787843a6d6f1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:42.442430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 563/1130] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:42 2022] GET /identity/v3/role_assignments?user.id=5854cb7ded52414b8050eb0806787f2f&scope.project.id=3058bf1ad7d74bcf9bff9120acb7f1c1&role.id=7673131020174664baa4d10a6f38ce2e => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:44.449174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad7d8-5b8e-4735-b4df-3a2e90a35f36 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:44.449734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad7d8-5b8e-4735-b4df-3a2e90a35f36 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:44.449734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ad7d8-5b8e-4735-b4df-3a2e90a35f36 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:44.451075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 568/1131] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:44.456122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c672453-e0c4-4cb6-8335-df6218d4f094 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:44.456508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c672453-e0c4-4cb6-8335-df6218d4f094 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:44.456679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c672453-e0c4-4cb6-8335-df6218d4f094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:44.473439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9c672453-e0c4-4cb6-8335-df6218d4f094 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:44.474604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9c672453-e0c4-4cb6-8335-df6218d4f094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:44.539324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 564/1132] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:44.562466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ad7d8-5b8e-4735-b4df-3a2e90a35f36 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:44.603426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:44.662018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ad7d8-5b8e-4735-b4df-3a2e90a35f36 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:44.694859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d7e2ca-1058-445d-98f1-4b4cf12211e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:44.695632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e2ca-1058-445d-98f1-4b4cf12211e0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:44.695936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e2ca-1058-445d-98f1-4b4cf12211e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:44.696207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d7e2ca-1058-445d-98f1-4b4cf12211e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:44.697930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7e2ca-1058-445d-98f1-4b4cf12211e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:44.699395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d7e2ca-1058-445d-98f1-4b4cf12211e0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:44.700957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 569/1133] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:44.706651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c672453-e0c4-4cb6-8335-df6218d4f094 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:44.739355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:44.801329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c672453-e0c4-4cb6-8335-df6218d4f094 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:44.834414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdedfb1e-aa6c-43e2-9a14-6364caa263e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:44.835068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedfb1e-aa6c-43e2-9a14-6364caa263e9 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:44.835353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedfb1e-aa6c-43e2-9a14-6364caa263e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:44.835629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdedfb1e-aa6c-43e2-9a14-6364caa263e9 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:44.836369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedfb1e-aa6c-43e2-9a14-6364caa263e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:44.837817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdedfb1e-aa6c-43e2-9a14-6364caa263e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:44.861503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 565/1134] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:14:44 2022] POST /identity/v3/projects => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:46.875972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f2ead-2803-4af4-b11d-2db9057f7f81 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:46.876477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f2ead-2803-4af4-b11d-2db9057f7f81 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:46.876585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f2ead-2803-4af4-b11d-2db9057f7f81 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:46.877873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 570/1135] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:46.882941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10afb82-57ad-4846-a220-a818345a1845 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:46.883319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10afb82-57ad-4846-a220-a818345a1845 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:46.883497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10afb82-57ad-4846-a220-a818345a1845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:46.900986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a10afb82-57ad-4846-a220-a818345a1845 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:46.902152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a10afb82-57ad-4846-a220-a818345a1845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:46.965537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 566/1136] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:14:46.989066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f2ead-2803-4af4-b11d-2db9057f7f81 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:47.087188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04f2ead-2803-4af4-b11d-2db9057f7f81 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:47.094129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:47.120206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e99fe-aac5-42cd-a604-cbf961850bd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:47.120982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e99fe-aac5-42cd-a604-cbf961850bd9 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:47.121262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e99fe-aac5-42cd-a604-cbf961850bd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:47.121533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e99fe-aac5-42cd-a604-cbf961850bd9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:47.122695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e99fe-aac5-42cd-a604-cbf961850bd9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:47.124156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e99fe-aac5-42cd-a604-cbf961850bd9 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:47.125735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 571/1137] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:14:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:47.131419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10afb82-57ad-4846-a220-a818345a1845 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:47.149513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:47.231002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10afb82-57ad-4846-a220-a818345a1845 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:47.264283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55248250-81fe-42e6-985c-89809c2a58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:47.264951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55248250-81fe-42e6-985c-89809c2a58c9 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:47.265237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55248250-81fe-42e6-985c-89809c2a58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:47.265515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55248250-81fe-42e6-985c-89809c2a58c9 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:47.266248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55248250-81fe-42e6-985c-89809c2a58c9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:47.267668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55248250-81fe-42e6-985c-89809c2a58c9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:47.286356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 567/1138] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:14:47 2022] POST /identity/v3/users => generated 286 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:14:49.303131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b3058-1e9d-458c-96ad-9da0ca5e4e78 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:49.303679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b3058-1e9d-458c-96ad-9da0ca5e4e78 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:49.303679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1b3058-1e9d-458c-96ad-9da0ca5e4e78 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:49.305213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 572/1139] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:49.311001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3a66-5b7d-41a2-b3c3-eb807e605b3d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:49.311397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3a66-5b7d-41a2-b3c3-eb807e605b3d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:49.311562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb3a66-5b7d-41a2-b3c3-eb807e605b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:49.328363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-11fb3a66-5b7d-41a2-b3c3-eb807e605b3d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:49.329542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-11fb3a66-5b7d-41a2-b3c3-eb807e605b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:49.392490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 568/1140] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:49.414843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b3058-1e9d-458c-96ad-9da0ca5e4e78 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:49.513830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1b3058-1e9d-458c-96ad-9da0ca5e4e78 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:49.546854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:49.547635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:49.547948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:49.548225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:49.558636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:49.566493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:49.567636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:49.568671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 573/1141] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:14:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:49.573564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3a66-5b7d-41a2-b3c3-eb807e605b3d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:49.576648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:49.670109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb3a66-5b7d-41a2-b3c3-eb807e605b3d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:49.703705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:49.704400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:49.704690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:49.704977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:49.705726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:49.706037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:49.707402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:49.715291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 569/1142] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:14:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:49.720820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:49.755044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:49.819112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c61813-f782-4342-8cec-95f53576fa81 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:49.851425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:49.852309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:49.852649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:49.853412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] PATH_INFO: `/v3/users/b79cf4f0a9d246dfa471c756a2ec4e41` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:49.866114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] RBAC: Authorizing `identity:get_user(user_id=b79cf4f0a9d246dfa471c756a2ec4e41)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:49.867540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:49.869241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 574/1143] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:49 2022] GET /identity/v3/users/b79cf4f0a9d246dfa471c756a2ec4e41 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:49.874890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:49.936826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:49.970196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa311de5-ba63-4ec3-896a-9315dedfac55 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:50.001738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:50.002368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:50.002648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:50.002923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:50.066780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:50.068235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:50.128251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:50.162961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 570/1144] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:50.169427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:50.260677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:50.269819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f9ff66-21c4-4409-9d33-760d7c3e081b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:50.303051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22f57b0-0e57-49ef-89b4-b910e9b8c5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:50.303925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f57b0-0e57-49ef-89b4-b910e9b8c5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:50.304468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f57b0-0e57-49ef-89b4-b910e9b8c5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:50.304468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22f57b0-0e57-49ef-89b4-b910e9b8c5e1 None admin] PATH_INFO: `/v3/projects/cccd2077e51a418287f98520b9a7fd3d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:50.305621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f57b0-0e57-49ef-89b4-b910e9b8c5e1 None admin] RBAC: Authorizing `identity:get_project(project_id=cccd2077e51a418287f98520b9a7fd3d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:50.307051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22f57b0-0e57-49ef-89b4-b910e9b8c5e1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:50.308759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 575/1145] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:50 2022] GET /identity/v3/projects/cccd2077e51a418287f98520b9a7fd3d => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:50.314992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:50.409894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839346cd-63d5-4b8c-9826-ce28122c2fa6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:50.442432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23155913-12d9-45ea-a3e3-4499a3681af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:50.443071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155913-12d9-45ea-a3e3-4499a3681af3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:50.443350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155913-12d9-45ea-a3e3-4499a3681af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:50.443620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23155913-12d9-45ea-a3e3-4499a3681af3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:50.444467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23155913-12d9-45ea-a3e3-4499a3681af3 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=cccd2077e51a418287f98520b9a7fd3d, user.id=b79cf4f0a9d246dfa471c756a2ec4e41) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:50.444778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23155913-12d9-45ea-a3e3-4499a3681af3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:50.446176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23155913-12d9-45ea-a3e3-4499a3681af3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:50.451936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 571/1146] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:50 2022] GET /identity/v3/role_assignments?user.id=b79cf4f0a9d246dfa471c756a2ec4e41&scope.project.id=cccd2077e51a418287f98520b9a7fd3d&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:52.408538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4409a3c-f05e-4f27-b6e2-00e78a01e31e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:52.409091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4409a3c-f05e-4f27-b6e2-00e78a01e31e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:52.409091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4409a3c-f05e-4f27-b6e2-00e78a01e31e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:52.410471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 576/1147] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:52.415714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255482c-dea2-405c-bb02-c0e47a49afae None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:52.416147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255482c-dea2-405c-bb02-c0e47a49afae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:52.416299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c255482c-dea2-405c-bb02-c0e47a49afae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:52.432995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c255482c-dea2-405c-bb02-c0e47a49afae None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:52.434422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c255482c-dea2-405c-bb02-c0e47a49afae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:52.457274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:52.499268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 572/1148] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:52.520965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4409a3c-f05e-4f27-b6e2-00e78a01e31e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:52.617872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4409a3c-f05e-4f27-b6e2-00e78a01e31e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:52.650956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:52.651725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:52.652031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:52.652303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:52.661982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:52.670042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:52.671129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:52.672198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 577/1149] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:14:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:52.677085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255482c-dea2-405c-bb02-c0e47a49afae None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:52.770835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c255482c-dea2-405c-bb02-c0e47a49afae None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:52.802912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:52.803562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:52.803878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:52.804153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:52.804890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:52.805194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:52.807161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:52.815103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 573/1150] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:14:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:52.820592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:52.842400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:52.916815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064570c5-d91c-420b-9146-5d008b63e38b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:52.948843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:52.949702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:52.949983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:52.950252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] PATH_INFO: `/v3/users/b79cf4f0a9d246dfa471c756a2ec4e41` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:52.951422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] RBAC: Authorizing `identity:get_user(user_id=b79cf4f0a9d246dfa471c756a2ec4e41)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:52.952884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:52.954475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 578/1151] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:52 2022] GET /identity/v3/users/b79cf4f0a9d246dfa471c756a2ec4e41 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:52.960028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:52.985177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:53.057969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb387bfc-0549-4bfe-8502-41e3ce2ed763 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:53.089986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aff02b4-0840-4ab3-a64a-8343064f830f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:53.090839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff02b4-0840-4ab3-a64a-8343064f830f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:53.091144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff02b4-0840-4ab3-a64a-8343064f830f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:53.091401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aff02b4-0840-4ab3-a64a-8343064f830f None admin] PATH_INFO: `/v3/projects/cccd2077e51a418287f98520b9a7fd3d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:53.092596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff02b4-0840-4ab3-a64a-8343064f830f None admin] RBAC: Authorizing `identity:get_project(project_id=cccd2077e51a418287f98520b9a7fd3d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:53.094003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aff02b4-0840-4ab3-a64a-8343064f830f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:53.095663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 574/1152] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:52 2022] GET /identity/v3/projects/cccd2077e51a418287f98520b9a7fd3d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:53.101619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:53.187346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:53.200035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539707bd-f277-4768-9d28-e61b89b7d6fb None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:53.233444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ccdc9-b3e0-48c6-95fd-78f07924b377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:53.234317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ccdc9-b3e0-48c6-95fd-78f07924b377 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:53.234597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ccdc9-b3e0-48c6-95fd-78f07924b377 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:53.234869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ccdc9-b3e0-48c6-95fd-78f07924b377 None admin] PATH_INFO: `/v3/projects/cccd2077e51a418287f98520b9a7fd3d/users/b79cf4f0a9d246dfa471c756a2ec4e41/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:53.236830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ccdc9-b3e0-48c6-95fd-78f07924b377 None admin] RBAC: Authorizing `identity:create_grant(project_id=cccd2077e51a418287f98520b9a7fd3d, user_id=b79cf4f0a9d246dfa471c756a2ec4e41, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:53.238248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ccdc9-b3e0-48c6-95fd-78f07924b377 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:53.250280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 579/1153] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:14:53 2022] PUT /identity/v3/projects/cccd2077e51a418287f98520b9a7fd3d/users/b79cf4f0a9d246dfa471c756a2ec4e41/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:14:55.235955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bfee-331e-46b8-a782-31bfb3d73ecd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:55.236485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bfee-331e-46b8-a782-31bfb3d73ecd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:55.236577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8435bfee-331e-46b8-a782-31bfb3d73ecd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:55.237876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 575/1154] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:55.243121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224bbb2-e9d4-474a-b061-483ce715ae7a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:55.243543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224bbb2-e9d4-474a-b061-483ce715ae7a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:55.243667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a224bbb2-e9d4-474a-b061-483ce715ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:55.260049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a224bbb2-e9d4-474a-b061-483ce715ae7a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:55.261220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a224bbb2-e9d4-474a-b061-483ce715ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:55.324966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 580/1155] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:55.347890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435bfee-331e-46b8-a782-31bfb3d73ecd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:55.435721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:55.446591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8435bfee-331e-46b8-a782-31bfb3d73ecd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:55.479026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:55.479852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:55.480144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:55.480426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:55.490702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:14:55.498279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:55.499354 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:14:55.500273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 576/1156] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:14:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:55.505477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224bbb2-e9d4-474a-b061-483ce715ae7a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:55.579359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:55.601014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a224bbb2-e9d4-474a-b061-483ce715ae7a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:55.632717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:55.633897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:55.634173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:55.634444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:55.635167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:55.635468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:55.636856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:55.644582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 581/1157] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:14:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:55.650867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:55.746590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8960892-0798-4286-b1dc-4240f4d6a401 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:55.779574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:55.780452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:55.780739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:55.781016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] PATH_INFO: `/v3/users/b79cf4f0a9d246dfa471c756a2ec4e41` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:55.782188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] RBAC: Authorizing `identity:get_user(user_id=b79cf4f0a9d246dfa471c756a2ec4e41)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:55.783616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:55.785246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 577/1158] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:14:55 2022] GET /identity/v3/users/b79cf4f0a9d246dfa471c756a2ec4e41 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:55.790634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:55.886396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a177046f-5561-4009-ad7a-c70adcec8f71 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:55.918834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:55.919458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:55.919762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:55.920048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:55.929866 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:55.984524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:55.985947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:56.078797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 582/1159] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:56.084629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:56.103862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:56.181907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e3050-fbb3-4d21-b401-eb08ce2ea615 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:56.214799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee08fe1-6187-4d4d-b43d-e0962c04d1e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:56.215636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08fe1-6187-4d4d-b43d-e0962c04d1e9 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:56.215944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08fe1-6187-4d4d-b43d-e0962c04d1e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:56.216215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee08fe1-6187-4d4d-b43d-e0962c04d1e9 None admin] PATH_INFO: `/v3/projects/cccd2077e51a418287f98520b9a7fd3d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:56.217365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee08fe1-6187-4d4d-b43d-e0962c04d1e9 None admin] RBAC: Authorizing `identity:get_project(project_id=cccd2077e51a418287f98520b9a7fd3d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:56.218772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee08fe1-6187-4d4d-b43d-e0962c04d1e9 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:56.220458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 578/1160] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:14:56 2022] GET /identity/v3/projects/cccd2077e51a418287f98520b9a7fd3d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:56.226428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:56.265707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:56.322136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06de224-22f7-40c3-ad94-63985bb54548 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:56.354717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff9e20-6f2c-4ab0-a832-5b2d19701a4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:56.355296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff9e20-6f2c-4ab0-a832-5b2d19701a4d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:56.355576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff9e20-6f2c-4ab0-a832-5b2d19701a4d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:56.355988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff9e20-6f2c-4ab0-a832-5b2d19701a4d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:56.356809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff9e20-6f2c-4ab0-a832-5b2d19701a4d None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=cccd2077e51a418287f98520b9a7fd3d, user.id=b79cf4f0a9d246dfa471c756a2ec4e41) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:56.357117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff9e20-6f2c-4ab0-a832-5b2d19701a4d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:56.358529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff9e20-6f2c-4ab0-a832-5b2d19701a4d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:56.364192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 583/1161] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:14:56 2022] GET /identity/v3/role_assignments?user.id=b79cf4f0a9d246dfa471c756a2ec4e41&scope.project.id=cccd2077e51a418287f98520b9a7fd3d&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:14:58.371119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51c6da-553e-4e3a-9fdf-f747ec108034 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:58.371722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51c6da-553e-4e3a-9fdf-f747ec108034 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:58.371722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac51c6da-553e-4e3a-9fdf-f747ec108034 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:58.373072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 579/1162] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:14:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:14:58.378232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c95-7360-41c8-bb4a-380d7093aacc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:58.378645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c95-7360-41c8-bb4a-380d7093aacc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:58.378787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26da1c95-7360-41c8-bb4a-380d7093aacc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:58.395728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-26da1c95-7360-41c8-bb4a-380d7093aacc None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:14:58.396928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-26da1c95-7360-41c8-bb4a-380d7093aacc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:14:58.460985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 584/1163] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:14:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:14:58.483467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51c6da-553e-4e3a-9fdf-f747ec108034 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:58.567487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:58.582929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac51c6da-553e-4e3a-9fdf-f747ec108034 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:58.616682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:58.617333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:58.617621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:58.617901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:58.682485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:58.684139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:58.762372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:58.777066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 580/1164] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:14:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:14:58.783045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1c95-7360-41c8-bb4a-380d7093aacc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:58.816329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:14:58.879628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26da1c95-7360-41c8-bb4a-380d7093aacc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:58.912149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52deb954-7de2-4723-aa61-1391975967c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:58.912930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deb954-7de2-4723-aa61-1391975967c2 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:58.913212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deb954-7de2-4723-aa61-1391975967c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:58.913484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52deb954-7de2-4723-aa61-1391975967c2 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:58.920514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52deb954-7de2-4723-aa61-1391975967c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:58.921953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52deb954-7de2-4723-aa61-1391975967c2 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-52deb954-7de2-4723-aa61-1391975967c2 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:14:58.936317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:14:58.937473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:14:58.938466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 585/1165] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:14:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:14:58.943335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:14:59.037586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537fa6e1-59ee-490a-ba01-af8505b31c66 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:14:59.069947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a7ab00-7e90-404c-a1bc-24ad3447e274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:14:59.070587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7ab00-7e90-404c-a1bc-24ad3447e274 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:14:59.070896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7ab00-7e90-404c-a1bc-24ad3447e274 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:14:59.071170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a7ab00-7e90-404c-a1bc-24ad3447e274 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:14:59.071942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7ab00-7e90-404c-a1bc-24ad3447e274 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:14:59.072248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7ab00-7e90-404c-a1bc-24ad3447e274 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:14:59.073587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a7ab00-7e90-404c-a1bc-24ad3447e274 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:14:59.080378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 581/1166] 10.222.0.16 () {56 vars in 1115 bytes} [Thu Jul 7 10:14:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:15:01.043465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835db18-76d0-48b0-b577-13b714cb105c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:01.044047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835db18-76d0-48b0-b577-13b714cb105c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:01.044047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835db18-76d0-48b0-b577-13b714cb105c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:01.045411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 586/1167] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:15:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:15:01.050377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3b252-286e-448f-9a3d-c784b7b6ee2d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:01.050768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3b252-286e-448f-9a3d-c784b7b6ee2d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:01.050936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3b252-286e-448f-9a3d-c784b7b6ee2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:01.067885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-39c3b252-286e-448f-9a3d-c784b7b6ee2d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:15:01.069053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-39c3b252-286e-448f-9a3d-c784b7b6ee2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:15:01.112883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:01.132994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 582/1168] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:15:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:15:01.155512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835db18-76d0-48b0-b577-13b714cb105c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:01.248241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:01.255970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835db18-76d0-48b0-b577-13b714cb105c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:01.288418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a8d395-4bcd-4e01-9a3b-72a5e454e8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:01.289072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d395-4bcd-4e01-9a3b-72a5e454e8dc None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:01.289353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d395-4bcd-4e01-9a3b-72a5e454e8dc None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:01.289629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a8d395-4bcd-4e01-9a3b-72a5e454e8dc None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:01.290391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8d395-4bcd-4e01-9a3b-72a5e454e8dc None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:01.291806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a8d395-4bcd-4e01-9a3b-72a5e454e8dc None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:01.305533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 587/1169] 10.222.0.16 () {60 vars in 1138 bytes} [Thu Jul 7 10:15:01 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:15:03.318692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc86c0c-24f5-42af-a7f7-66143a1b6370 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:03.319227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc86c0c-24f5-42af-a7f7-66143a1b6370 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:03.319227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc86c0c-24f5-42af-a7f7-66143a1b6370 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:03.320782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 583/1170] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:15:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:15:03.325886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eddded-9817-4225-9cc2-fef99d9b26b4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:03.326289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eddded-9817-4225-9cc2-fef99d9b26b4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:03.326451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eddded-9817-4225-9cc2-fef99d9b26b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:03.343574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-45eddded-9817-4225-9cc2-fef99d9b26b4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:15:03.344785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45eddded-9817-4225-9cc2-fef99d9b26b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:15:03.409226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 588/1171] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:15:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:15:03.432747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc86c0c-24f5-42af-a7f7-66143a1b6370 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:03.535623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc86c0c-24f5-42af-a7f7-66143a1b6370 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:03.569120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7361ec6c-dff5-4690-b075-8ee0612a8ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:03.569914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ec6c-dff5-4690-b075-8ee0612a8ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:03.570195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ec6c-dff5-4690-b075-8ee0612a8ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:03.570904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7361ec6c-dff5-4690-b075-8ee0612a8ae3 None admin] PATH_INFO: `/v3/domains/026054c0b41746f29eb07854e4e01a02` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:03.580996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361ec6c-dff5-4690-b075-8ee0612a8ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=026054c0b41746f29eb07854e4e01a02)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:03.582469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7361ec6c-dff5-4690-b075-8ee0612a8ae3 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:03.584131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 584/1172] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Jul 7 10:15:03 2022] GET /identity/v3/domains/026054c0b41746f29eb07854e4e01a02 => generated 267 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:03.590530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eddded-9817-4225-9cc2-fef99d9b26b4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:03.672922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:03.690167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eddded-9817-4225-9cc2-fef99d9b26b4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:03.723738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec9673-c699-4f0a-80bd-8ed74fd113ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:03.724887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec9673-c699-4f0a-80bd-8ed74fd113ff None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:03.725193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec9673-c699-4f0a-80bd-8ed74fd113ff None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:03.725469 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec9673-c699-4f0a-80bd-8ed74fd113ff None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:03.726177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec9673-c699-4f0a-80bd-8ed74fd113ff None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:03.727604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec9673-c699-4f0a-80bd-8ed74fd113ff None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:03.753067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 589/1173] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:15:03 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:15:05.771144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5030d2-aab4-4924-8ef5-0cfce501161d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:05.771663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5030d2-aab4-4924-8ef5-0cfce501161d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:05.771663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5030d2-aab4-4924-8ef5-0cfce501161d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:05.773026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 585/1174] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:15:05.778762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389412-7aaf-46ac-9ad7-313785f930f7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:05.779153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389412-7aaf-46ac-9ad7-313785f930f7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:05.779312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee389412-7aaf-46ac-9ad7-313785f930f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:05.796333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ee389412-7aaf-46ac-9ad7-313785f930f7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:15:05.797531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ee389412-7aaf-46ac-9ad7-313785f930f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:15:05.862788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 590/1175] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:15:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:15:05.888053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5030d2-aab4-4924-8ef5-0cfce501161d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:05.929925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:05.986878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5030d2-aab4-4924-8ef5-0cfce501161d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:06.018513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1507850d-9178-4c27-97f7-625c786bb59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:06.019358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507850d-9178-4c27-97f7-625c786bb59f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:06.019641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507850d-9178-4c27-97f7-625c786bb59f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:06.019940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1507850d-9178-4c27-97f7-625c786bb59f None admin] PATH_INFO: `/v3/domains/026054c0b41746f29eb07854e4e01a02` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:06.021098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507850d-9178-4c27-97f7-625c786bb59f None admin] RBAC: Authorizing `identity:get_domain(domain_id=026054c0b41746f29eb07854e4e01a02)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:06.022551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1507850d-9178-4c27-97f7-625c786bb59f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:06.024118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 586/1176] 10.222.0.16 () {56 vars in 1183 bytes} [Thu Jul 7 10:15:05 2022] GET /identity/v3/domains/026054c0b41746f29eb07854e4e01a02 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:06.030422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389412-7aaf-46ac-9ad7-313785f930f7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:06.094949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:06.128870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee389412-7aaf-46ac-9ad7-313785f930f7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:06.162095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca09e8a-025c-46dd-8918-e4c93b388682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:06.162758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca09e8a-025c-46dd-8918-e4c93b388682 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:06.163042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca09e8a-025c-46dd-8918-e4c93b388682 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:06.163319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca09e8a-025c-46dd-8918-e4c93b388682 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:06.164050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca09e8a-025c-46dd-8918-e4c93b388682 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:06.165518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca09e8a-025c-46dd-8918-e4c93b388682 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:06.184616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 591/1177] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:15:06 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:15:08.200729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bbf05-a887-4298-a354-71cb3a4bea4f None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:08.201285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bbf05-a887-4298-a354-71cb3a4bea4f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:08.201285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385bbf05-a887-4298-a354-71cb3a4bea4f None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:08.202644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 587/1178] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:15:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:15:08.207609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c45cc7-e95e-409f-8fd1-6801bc86b8b1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:08.208065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c45cc7-e95e-409f-8fd1-6801bc86b8b1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:08.208196 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c45cc7-e95e-409f-8fd1-6801bc86b8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:08.225069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-84c45cc7-e95e-409f-8fd1-6801bc86b8b1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:15:08.226246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-84c45cc7-e95e-409f-8fd1-6801bc86b8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:15:08.290702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 592/1179] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:15:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:15:08.313949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bbf05-a887-4298-a354-71cb3a4bea4f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:08.411715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385bbf05-a887-4298-a354-71cb3a4bea4f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:08.420093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:08.444129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:08.444913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:08.445223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:08.445500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:08.456540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:15:08.464678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:15:08.465645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:15:08.466589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 588/1180] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:15:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:15:08.471860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c45cc7-e95e-409f-8fd1-6801bc86b8b1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:08.521883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:08.568510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c45cc7-e95e-409f-8fd1-6801bc86b8b1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:08.601027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:08.601687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:08.601967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:08.602240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:08.602973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:15:08.603276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:08.604663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:08.612501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 593/1181] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:15:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:08.618632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:08.713070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f73d8f5-c21e-4294-9b51-9805b79c1365 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:08.744980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:08.745863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:08.746142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:08.746411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] PATH_INFO: `/v3/users/2a70e2a0d3f94b6b97ece39e6489e108` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:08.759285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] RBAC: Authorizing `identity:get_user(user_id=2a70e2a0d3f94b6b97ece39e6489e108)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:08.760723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:08.762321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 589/1182] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:15:08 2022] GET /identity/v3/users/2a70e2a0d3f94b6b97ece39e6489e108 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:08.767758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:08.862347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6633dc-f141-4796-90b1-56485c535c6e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:08.875576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:08.895382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:08.896040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:08.896317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:08.896588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:08.960640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:08.962089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:09.054325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 594/1183] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:15:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:15:09.060780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:09.086833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:09.156045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35d640-5021-4766-8548-fa3bcfd69be6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:09.190457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f799f-e84d-494b-a76e-e9c4304e90dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:09.191286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f799f-e84d-494b-a76e-e9c4304e90dd None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:09.191566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f799f-e84d-494b-a76e-e9c4304e90dd None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:09.191867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f799f-e84d-494b-a76e-e9c4304e90dd None admin] PATH_INFO: `/v3/projects/bf22e02b0a7b48a88ad65efa630667f4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:09.193021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f799f-e84d-494b-a76e-e9c4304e90dd None admin] RBAC: Authorizing `identity:get_project(project_id=bf22e02b0a7b48a88ad65efa630667f4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:09.194417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f799f-e84d-494b-a76e-e9c4304e90dd None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:09.196104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 590/1184] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:15:09 2022] GET /identity/v3/projects/bf22e02b0a7b48a88ad65efa630667f4 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:09.202080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:09.216315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:09.302419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba5edb9-bd6b-4ae3-98c4-3b9de45dec6e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:09.335070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279f1608-76a1-4305-96e0-f4bac9284dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:09.335707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1608-76a1-4305-96e0-f4bac9284dad None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:09.336017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1608-76a1-4305-96e0-f4bac9284dad None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:09.336290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279f1608-76a1-4305-96e0-f4bac9284dad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:09.337616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f1608-76a1-4305-96e0-f4bac9284dad None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=bf22e02b0a7b48a88ad65efa630667f4, user.id=2a70e2a0d3f94b6b97ece39e6489e108) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:15:09.337924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f1608-76a1-4305-96e0-f4bac9284dad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:09.339336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279f1608-76a1-4305-96e0-f4bac9284dad None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:09.344700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 595/1185] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:15:09 2022] GET /identity/v3/role_assignments?user.id=2a70e2a0d3f94b6b97ece39e6489e108&scope.project.id=bf22e02b0a7b48a88ad65efa630667f4&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:11.309773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd382-cdfb-4027-9a07-2de144b0f2db None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:11.310328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd382-cdfb-4027-9a07-2de144b0f2db None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:11.310328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4fd382-cdfb-4027-9a07-2de144b0f2db None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:11.311663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 591/1186] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:15:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:15:11.316968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d512364-feae-4ad8-98ab-37d3fc1419a0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:11.317409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d512364-feae-4ad8-98ab-37d3fc1419a0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:11.317530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d512364-feae-4ad8-98ab-37d3fc1419a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:11.333830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7d512364-feae-4ad8-98ab-37d3fc1419a0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:15:11.335011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7d512364-feae-4ad8-98ab-37d3fc1419a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:15:11.398736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 596/1187] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:15:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:15:11.421031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fd382-cdfb-4027-9a07-2de144b0f2db None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:11.511248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:11.519001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4fd382-cdfb-4027-9a07-2de144b0f2db None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:11.551837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:11.552568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:11.552849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:11.553122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:11.563527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:15:11.571259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:15:11.572401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:15:11.573381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 592/1188] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:15:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:15:11.578565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d512364-feae-4ad8-98ab-37d3fc1419a0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:11.589108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:11.675251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d512364-feae-4ad8-98ab-37d3fc1419a0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:11.708291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:11.708945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:11.709477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:11.709760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:11.710503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:15:11.710809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:11.712232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:11.719969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 597/1189] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:15:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:11.725509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:11.821613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce8787e-9c6c-460e-8acf-eaec8b8885c2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:11.853146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:11.854006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:11.854288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:11.854563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] PATH_INFO: `/v3/users/2a70e2a0d3f94b6b97ece39e6489e108` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:11.855726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] RBAC: Authorizing `identity:get_user(user_id=2a70e2a0d3f94b6b97ece39e6489e108)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:11.857148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:11.858747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 593/1190] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:15:11 2022] GET /identity/v3/users/2a70e2a0d3f94b6b97ece39e6489e108 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:11.864442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:11.932543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:11.963404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a22e56c-49dc-4ed0-8889-c9a114080263 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:11.995895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660130d5-2292-4ccb-a706-5238f8e3fc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:11.996739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660130d5-2292-4ccb-a706-5238f8e3fc25 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:11.997021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660130d5-2292-4ccb-a706-5238f8e3fc25 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:11.997295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660130d5-2292-4ccb-a706-5238f8e3fc25 None admin] PATH_INFO: `/v3/projects/bf22e02b0a7b48a88ad65efa630667f4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:11.998457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660130d5-2292-4ccb-a706-5238f8e3fc25 None admin] RBAC: Authorizing `identity:get_project(project_id=bf22e02b0a7b48a88ad65efa630667f4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:11.999906 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660130d5-2292-4ccb-a706-5238f8e3fc25 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:12.001569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 598/1191] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:15:11 2022] GET /identity/v3/projects/bf22e02b0a7b48a88ad65efa630667f4 => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:12.007227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:12.031035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:12.103046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a912fd-9a31-4b99-9f9f-2d9d67a48644 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:12.134665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a837484b-04a3-4c1c-9552-58902c24a195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:12.135532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837484b-04a3-4c1c-9552-58902c24a195 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:12.135838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837484b-04a3-4c1c-9552-58902c24a195 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:12.136112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a837484b-04a3-4c1c-9552-58902c24a195 None admin] PATH_INFO: `/v3/projects/bf22e02b0a7b48a88ad65efa630667f4/users/2a70e2a0d3f94b6b97ece39e6489e108/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:12.138014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837484b-04a3-4c1c-9552-58902c24a195 None admin] RBAC: Authorizing `identity:create_grant(project_id=bf22e02b0a7b48a88ad65efa630667f4, user_id=2a70e2a0d3f94b6b97ece39e6489e108, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:12.139410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a837484b-04a3-4c1c-9552-58902c24a195 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:12.151675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 594/1192] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:15:12 2022] PUT /identity/v3/projects/bf22e02b0a7b48a88ad65efa630667f4/users/2a70e2a0d3f94b6b97ece39e6489e108/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:15:14.131578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7561f6-04d5-47c0-b75a-ce90203679a9 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:14.132162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7561f6-04d5-47c0-b75a-ce90203679a9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:14.132162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7561f6-04d5-47c0-b75a-ce90203679a9 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:14.134312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 599/1193] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:15:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:15:14.139376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea730d-e6b2-4588-b188-840850971651 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:14.139768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea730d-e6b2-4588-b188-840850971651 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:14.139969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ea730d-e6b2-4588-b188-840850971651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:14.156337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a5ea730d-e6b2-4588-b188-840850971651 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:15:14.157532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a5ea730d-e6b2-4588-b188-840850971651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:15:14.222115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 595/1194] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:15:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:15:14.246105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7561f6-04d5-47c0-b75a-ce90203679a9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:14.343799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7561f6-04d5-47c0-b75a-ce90203679a9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:14.375959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:14.376722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:14.377000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:14.377272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:14.383954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:14.388528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:15:14.396319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:15:14.397401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:15:14.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 600/1195] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:15:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:15:14.403400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea730d-e6b2-4588-b188-840850971651 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:14.418623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:14.499753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ea730d-e6b2-4588-b188-840850971651 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:14.532150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:14.532820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:14.533105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:14.533383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:14.534132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:15:14.534438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:14.535828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:14.544107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 596/1196] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:15:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:14.549851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:14.645257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a01d94-6efd-4493-80f4-fc3ea264a396 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:14.678094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:14.678936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:14.679213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:14.679483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] PATH_INFO: `/v3/users/2a70e2a0d3f94b6b97ece39e6489e108` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:14.680694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] RBAC: Authorizing `identity:get_user(user_id=2a70e2a0d3f94b6b97ece39e6489e108)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:14.682112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:14.683981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 601/1197] 10.222.0.16 () {56 vars in 1177 bytes} [Thu Jul 7 10:15:14 2022] GET /identity/v3/users/2a70e2a0d3f94b6b97ece39e6489e108 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:14.689412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:14.761925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:14.786249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2098449-3d6a-4a10-a504-f2cc347ca097 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:14.818484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:14.819115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:14.819399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:14.819673 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:14.883761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:14.885198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:14.954543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:14.979991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 597/1198] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:15:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 10:15:14.985734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:15.047829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:15.081602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f12080-b3d7-4d85-a6fb-e0933cb4797f None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:15.113914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a26ef82-e82a-4d3b-b0d8-2e2aacda793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:15.114793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26ef82-e82a-4d3b-b0d8-2e2aacda793e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:15.115074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26ef82-e82a-4d3b-b0d8-2e2aacda793e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:15.115346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a26ef82-e82a-4d3b-b0d8-2e2aacda793e None admin] PATH_INFO: `/v3/projects/bf22e02b0a7b48a88ad65efa630667f4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:15.116526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26ef82-e82a-4d3b-b0d8-2e2aacda793e None admin] RBAC: Authorizing `identity:get_project(project_id=bf22e02b0a7b48a88ad65efa630667f4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:15.118489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a26ef82-e82a-4d3b-b0d8-2e2aacda793e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:15.120284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 602/1199] 10.222.0.16 () {56 vars in 1186 bytes} [Thu Jul 7 10:15:14 2022] GET /identity/v3/projects/bf22e02b0a7b48a88ad65efa630667f4 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:15.126104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:15:15.219438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1408b45-b0ce-4af0-b9fa-8faf347b1ea2 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:15:15.251969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500e7e29-97b4-48fa-af41-0cdf9e3cf3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:15:15.252602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7e29-97b4-48fa-af41-0cdf9e3cf3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:15:15.252893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7e29-97b4-48fa-af41-0cdf9e3cf3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:15:15.253154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500e7e29-97b4-48fa-af41-0cdf9e3cf3d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:15:15.253975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7e29-97b4-48fa-af41-0cdf9e3cf3d6 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=bf22e02b0a7b48a88ad65efa630667f4, user.id=2a70e2a0d3f94b6b97ece39e6489e108) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:15:15.254287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7e29-97b4-48fa-af41-0cdf9e3cf3d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:15:15.255678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500e7e29-97b4-48fa-af41-0cdf9e3cf3d6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:15:15.261279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 598/1200] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:15:15 2022] GET /identity/v3/role_assignments?user.id=2a70e2a0d3f94b6b97ece39e6489e108&scope.project.id=bf22e02b0a7b48a88ad65efa630667f4&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:15:25.109897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=33 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:15:25.258719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:09.978351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee807-f32a-444f-871f-d2c0ff13b4b3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:09.979023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee807-f32a-444f-871f-d2c0ff13b4b3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:09.979023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ee807-f32a-444f-871f-d2c0ff13b4b3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:09.980587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 603/1201] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:09.985749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad94005-71f7-4d79-9c80-1a7ba85a0d6c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:09.986294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad94005-71f7-4d79-9c80-1a7ba85a0d6c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:09.986294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad94005-71f7-4d79-9c80-1a7ba85a0d6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:10.003489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cad94005-71f7-4d79-9c80-1a7ba85a0d6c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:10.005315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cad94005-71f7-4d79-9c80-1a7ba85a0d6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:10.100067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 599/1202] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:09 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:10.128486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee807-f32a-444f-871f-d2c0ff13b4b3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:10.258780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ee807-f32a-444f-871f-d2c0ff13b4b3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:10.308595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:10.309522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:10.309944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:10.310309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:10.323054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:10.324741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:10.344511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:10.345620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:10.346607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:10.346607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:10.346607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:10.346607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:10.346607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 604/1203] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:10.352301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad94005-71f7-4d79-9c80-1a7ba85a0d6c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:10.492096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad94005-71f7-4d79-9c80-1a7ba85a0d6c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:10.544863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:10.552858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765eae2-1a5a-4606-80e4-c009a1b3db63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:10.553634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765eae2-1a5a-4606-80e4-c009a1b3db63 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:10.553942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765eae2-1a5a-4606-80e4-c009a1b3db63 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:10.554246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765eae2-1a5a-4606-80e4-c009a1b3db63 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:10.555222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765eae2-1a5a-4606-80e4-c009a1b3db63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:10.555587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765eae2-1a5a-4606-80e4-c009a1b3db63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:10.557552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765eae2-1a5a-4606-80e4-c009a1b3db63 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:10.580039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 600/1204] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:16:10.581422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:10.661646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:10.720974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483a59db-24c4-4ee0-8e87-352e2314e701 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:10.764314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135731f6-2acc-4299-b4ac-1c253edf2f47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:10.765107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135731f6-2acc-4299-b4ac-1c253edf2f47 None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:10.765394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135731f6-2acc-4299-b4ac-1c253edf2f47 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:10.765672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135731f6-2acc-4299-b4ac-1c253edf2f47 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:10.766423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135731f6-2acc-4299-b4ac-1c253edf2f47 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:10.768027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135731f6-2acc-4299-b4ac-1c253edf2f47 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:10.791965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 605/1205] 10.222.0.16 () {60 vars in 1132 bytes} [Thu Jul 7 10:16:10 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:16:13.014728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f884-564f-4406-addb-4c1574fe29ce None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:13.015286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f884-564f-4406-addb-4c1574fe29ce None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:13.015464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3f884-564f-4406-addb-4c1574fe29ce None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:13.017113 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 601/1206] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:13.025586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d478536-e30d-428d-a7d2-b87b11c5bfcb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:13.025586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d478536-e30d-428d-a7d2-b87b11c5bfcb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:13.025586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d478536-e30d-428d-a7d2-b87b11c5bfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:13.042939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8d478536-e30d-428d-a7d2-b87b11c5bfcb None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:13.044177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8d478536-e30d-428d-a7d2-b87b11c5bfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:13.122339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 606/1207] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 07 10:16:13.147114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f884-564f-4406-addb-4c1574fe29ce None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:13.269407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3f884-564f-4406-addb-4c1574fe29ce None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:13.307316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:13.308469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:13.308768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:13.309029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:13.325859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:16:13.335024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:13.336232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:16:13.337405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:13.339920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 602/1208] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:16:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:13.343021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d478536-e30d-428d-a7d2-b87b11c5bfcb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:13.384796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:13.448535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d478536-e30d-428d-a7d2-b87b11c5bfcb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:13.484100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:13.484805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:13.485147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:13.485404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:13.486288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:13.486602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:13.488135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:13.496701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 607/1209] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:16:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:13.503146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:13.567641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:13.626818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271df294-4e4c-4836-896a-ae0617db39fb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:13.670355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:13.671277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:13.671567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:13.671874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:13.683348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:13.685083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:13.705854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:13.706973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:13.708711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:13.708711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:13.708711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:13.708711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:13.710605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 603/1210] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:13.716463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:13.825496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:13.827702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ba55d-c073-456a-9d34-6130cc57eff6 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:13.865632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:13.866436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:13.866673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:13.866966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:13.867677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:13.868801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:13.869346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:13.882765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 608/1211] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:13.888572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:14.006164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1ba3f-37d7-497d-9728-9ffe7ec6f632 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:14.008248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:14.050141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:14.051005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:14.051289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:14.051563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:14.062237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:14.064123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:16:14.080784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:14.082538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:14.084890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 604/1212] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:14.088565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:14.205043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140611c-ffe5-4dd1-8b84-b030f87e5fe5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:14.247370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:14.248269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:14.248545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:14.248869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:14.250219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:14.250620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:14.252492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:14.277184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 609/1213] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jul 7 10:16:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:14.286445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:14.412884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5efcc90-754e-45b8-9693-e8130b2b78ab None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:14.462588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:14.463272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:14.463562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:14.463876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:14.496068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:14.541900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:14.543373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:14.662265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 605/1214] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:16:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:16:14.663488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:14.685414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:14.775420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559ef39-0bc7-4bf5-98b1-70cb3694080d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:14.817068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:14.817953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:14.818236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:14.818511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:14.830065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:14.831634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:14.851895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:14.852936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:14.854405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:14.854405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:14.854405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:14.854405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:14.854405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 610/1215] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:14.858695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:14.934548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:14.977029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72aa1b0c-863e-4117-bee7-1a2c02333e8d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:15.012733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:15.013492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:15.013845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:15.014191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:15.014998 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:15.015379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:15.016846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:15.030336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 606/1216] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:15.036410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:15.127404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:15.170540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b134bd-5c99-44fd-a1c9-e04e860b338e None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:15.206490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:15.207791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:15.208063 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:15.208334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:15.215836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:15.217305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:16:15.231052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:15.232250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:15.238222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 611/1217] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:16:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:15.241460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:15.372631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43170df-327a-4add-9b09-73d53b8a303a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:15.403640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:15.416350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb9e5e9-0ca3-49e6-bfe1-6b3d36cedd7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:15.417083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9e5e9-0ca3-49e6-bfe1-6b3d36cedd7e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:15.417436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9e5e9-0ca3-49e6-bfe1-6b3d36cedd7e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:15.417777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb9e5e9-0ca3-49e6-bfe1-6b3d36cedd7e None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:15.418624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9e5e9-0ca3-49e6-bfe1-6b3d36cedd7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:15.419003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9e5e9-0ca3-49e6-bfe1-6b3d36cedd7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:15.420566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb9e5e9-0ca3-49e6-bfe1-6b3d36cedd7e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:15.435195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 607/1218] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:16:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:15.442160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:15.571597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b35998e-68e7-4c74-a31f-39d79bdb11a0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:15.603025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:15.609501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83061863-7921-463a-952d-e6d65936f24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:15.610353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061863-7921-463a-952d-e6d65936f24f None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:15.610643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061863-7921-463a-952d-e6d65936f24f None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:15.610959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83061863-7921-463a-952d-e6d65936f24f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:15.611926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83061863-7921-463a-952d-e6d65936f24f None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=a0eae5ef8c1747408cbaad55be572987) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:15.612240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83061863-7921-463a-952d-e6d65936f24f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:15.614220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83061863-7921-463a-952d-e6d65936f24f None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:15.621176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 612/1219] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:16:15 2022] GET /identity/v3/role_assignments?user.id=a0eae5ef8c1747408cbaad55be572987&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:17.726340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed727-7adb-44b1-8593-f7f7aa2e7d49 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:17.726865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed727-7adb-44b1-8593-f7f7aa2e7d49 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:17.726987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ed727-7adb-44b1-8593-f7f7aa2e7d49 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:17.728421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 608/1220] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:17.733614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea7ab3-8083-4fbd-9abb-5524a4b05667 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:17.734015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea7ab3-8083-4fbd-9abb-5524a4b05667 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:17.734170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ea7ab3-8083-4fbd-9abb-5524a4b05667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:17.751262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-60ea7ab3-8083-4fbd-9abb-5524a4b05667 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:17.752499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-60ea7ab3-8083-4fbd-9abb-5524a4b05667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:17.824655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 613/1221] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:17.850180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ed727-7adb-44b1-8593-f7f7aa2e7d49 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:17.964058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ed727-7adb-44b1-8593-f7f7aa2e7d49 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:17.998266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:17.999060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:17.999349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:17.999624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:18.009962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:16:18.018478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:18.020271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:18.021271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 609/1222] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:16:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:16:18.027383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea7ab3-8083-4fbd-9abb-5524a4b05667 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:18.133020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ea7ab3-8083-4fbd-9abb-5524a4b05667 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:18.156311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:18.167813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:18.168481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:18.168759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:18.169030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:18.169762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] RBAC: Adding query filter params (name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:18.170064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:18.171419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:18.179449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 614/1223] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:16:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:18.185618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:18.231427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:18.293759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f48d0-5ed9-436f-91af-7b067696572d None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:18.330357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:18.331154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:18.331441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:18.331762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:18.342649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:18.344124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:18.363613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:18.364735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:18.365838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:18.365838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:18.365838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:18.365838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:18.370358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 610/1224] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:18.371500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:18.476243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3259aab-5b9e-4312-8580-732dd10d4569 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:18.512790 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:18.513792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:18.514079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:18.514354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:18.516130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:18.516130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:18.517090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:18.529402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 615/1225] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:18.536091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:18.636612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:18.643976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a45915a-d189-49a9-8bab-dff38e1fd452 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:18.681052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:18.681885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:18.682166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:18.682441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:18.691054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:18.692512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:16:18.706915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:18.708251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:18.709447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 611/1226] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:16:18.714152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:18.764749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:18.823678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336404db-3c73-4756-8dbf-b7089cc530ee None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:18.861915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:18.862601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:18.863546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:18.863801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:18.864667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:18.865014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:18.866445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:18.885092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 616/1227] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jul 7 10:16:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:16:18.890744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:19.008041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05be7126-88fa-4bb3-80df-516d41d8aafb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:19.044559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:19.045440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:19.045770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:19.046087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:19.059162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:19.060636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:19.079885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:19.080897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:19.082716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:19.082716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:19.082716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:19.082716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:19.082716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 612/1228] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:19.087483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:19.190578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cfb446-2e41-402d-b197-64342e377bf1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:19.192938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:19.228361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:19.229013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:19.229297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:19.229567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:19.230269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:19.230578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:19.231966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:19.245055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 617/1229] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:19.251289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:19.262257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:19.365402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e04603-c3fd-48f4-a461-a0b9bb09c30a None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:19.403939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:19.404744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:19.405032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:19.405305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:19.412859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:19.414291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:16:19.429663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:19.431000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:19.433050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 613/1230] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:16:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:19.437662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:19.557342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2d28a9-7d61-4c93-aa5e-13defe2df1c5 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:19.594388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c57c9d3-194c-437b-acd4-b2010906ff58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:19.595124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57c9d3-194c-437b-acd4-b2010906ff58 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:19.595408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57c9d3-194c-437b-acd4-b2010906ff58 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:19.595687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c57c9d3-194c-437b-acd4-b2010906ff58 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:19.596517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57c9d3-194c-437b-acd4-b2010906ff58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:19.596885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57c9d3-194c-437b-acd4-b2010906ff58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:19.598329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c57c9d3-194c-437b-acd4-b2010906ff58 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:19.610931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 618/1231] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:16:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:16:19.617677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:19.688692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:19.726632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e0accc-538c-4d75-86c2-fa63820fd21c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:19.761774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c96fa7-1d44-410e-90d1-27fe78f88914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:19.762818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c96fa7-1d44-410e-90d1-27fe78f88914 None admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:19.763101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c96fa7-1d44-410e-90d1-27fe78f88914 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:19.763379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c96fa7-1d44-410e-90d1-27fe78f88914 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/a0eae5ef8c1747408cbaad55be572987/roles/54eab69c526d4d53807c7d0f9b1862c7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:19.780525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c96fa7-1d44-410e-90d1-27fe78f88914 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=a0eae5ef8c1747408cbaad55be572987, role_id=54eab69c526d4d53807c7d0f9b1862c7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:19.782063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c96fa7-1d44-410e-90d1-27fe78f88914 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:19.794626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 614/1232] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:16:19 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/a0eae5ef8c1747408cbaad55be572987/roles/54eab69c526d4d53807c7d0f9b1862c7 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:16:21.889050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0eab4-c0ab-48b5-ac85-c57077b103a5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:21.889638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0eab4-c0ab-48b5-ac85-c57077b103a5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:21.889753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0eab4-c0ab-48b5-ac85-c57077b103a5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:21.891298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 619/1233] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:21.896907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fcc08-f1ba-40a2-a26c-2dfff25a80a9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:21.897334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fcc08-f1ba-40a2-a26c-2dfff25a80a9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:21.897469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fcc08-f1ba-40a2-a26c-2dfff25a80a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:21.915322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a28fcc08-f1ba-40a2-a26c-2dfff25a80a9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:21.916759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a28fcc08-f1ba-40a2-a26c-2dfff25a80a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:21.987412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 615/1234] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:22.011123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0eab4-c0ab-48b5-ac85-c57077b103a5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:22.038702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:22.123575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0eab4-c0ab-48b5-ac85-c57077b103a5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:22.165434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:22.166162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:22.166446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:22.166721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:22.178120 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:16:22.186792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:22.188051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:22.188902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 620/1235] 10.222.0.16 () {56 vars in 1102 bytes} [Thu Jul 7 10:16:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:22.194249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fcc08-f1ba-40a2-a26c-2dfff25a80a9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:22.245181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:22.314597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fcc08-f1ba-40a2-a26c-2dfff25a80a9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:22.351322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:22.352041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:22.352355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:22.352609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:22.353359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] RBAC: Adding query filter params (name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:22.353668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:22.355038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:22.363902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 616/1236] 10.222.0.16 () {56 vars in 1103 bytes} [Thu Jul 7 10:16:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:16:22.369749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:22.459254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:22.476471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdd5497-ad12-4012-a702-90498d007d0b None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:22.511887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:22.512686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:22.512965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:22.513240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:22.525089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:22.527278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:22.550092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:22.551197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:22.552365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:22.552365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:22.552365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:22.552365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:22.552365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 621/1237] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:22.558819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:22.669975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58132f-e597-4ae9-84a1-3005edd1752c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:22.679552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:22.708285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:22.708955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:22.709741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:22.709988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:22.710770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:22.711080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:22.712471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:22.727050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 617/1238] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:22.732778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:22.838159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f64dc0a-8fa8-41ea-9322-3a3d2f593647 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:22.871664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:22.872488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:22.872765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:22.873053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:22.875783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:22.881721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:22.883303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:16:22.900894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:22.903049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:22.904925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 622/1239] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:22.911124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:23.029208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2321efed-b3ea-4023-8a1f-d763780279a0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:23.067973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:23.068667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:23.068965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:23.069210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:23.070057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:23.070352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:23.071796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:23.089276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 618/1240] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jul 7 10:16:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:16:23.094975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:23.202479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49155516-ab83-4c0d-af50-4e71185b15e1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:23.241609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:23.242322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:23.242609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:23.242890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:23.318640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:23.326188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:23.327706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:23.430428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 623/1241] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:16:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:16:23.438027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:23.468424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:23.549170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cf6997-a68b-4051-9083-22b9347c238c None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:23.585102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:23.585890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:23.586237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:23.586538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:23.598095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:23.599667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:23.618743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:23.620005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:23.620921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:23.620921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:23.620921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:23.620921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:23.621842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 619/1242] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:23.627888 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:23.730641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:23.738080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb9e76-dda8-4971-a05b-9d1dc49fdbf0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:23.773780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:23.774446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:23.774722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:23.774995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:23.775700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:23.776048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:23.777404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:23.789981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 624/1243] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:23.795817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:23.883456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:23.906183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa72e2f-32aa-4fa9-b6d3-07b387ba11e7 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:23.943088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:23.943929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:23.944225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:23.944509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:23.952160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:23.954083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:16:23.969847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:23.970844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:23.972045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 620/1244] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:16:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:16:23.977042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:24.089615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0709f7-d494-4f38-8b8f-1f3b703d53d0 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:24.125651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f28e7a2-8323-4463-b98e-22e7f6746b76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:24.126306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e7a2-8323-4463-b98e-22e7f6746b76 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:24.126586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e7a2-8323-4463-b98e-22e7f6746b76 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:24.126860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28e7a2-8323-4463-b98e-22e7f6746b76 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:24.127636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28e7a2-8323-4463-b98e-22e7f6746b76 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:24.127994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28e7a2-8323-4463-b98e-22e7f6746b76 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:24.129963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f28e7a2-8323-4463-b98e-22e7f6746b76 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:24.142916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 625/1245] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:16:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:24.150397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:24.267547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2914940-4af6-4cd4-b099-af25652963de None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:24.309360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c0f924-a513-4847-9617-cc7a99feb57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:24.310347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f924-a513-4847-9617-cc7a99feb57e None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:24.310647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f924-a513-4847-9617-cc7a99feb57e None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:24.311018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c0f924-a513-4847-9617-cc7a99feb57e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:24.312083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0f924-a513-4847-9617-cc7a99feb57e None admin] RBAC: Adding query filter params (role.id=54eab69c526d4d53807c7d0f9b1862c7, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=a0eae5ef8c1747408cbaad55be572987) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:24.312476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0f924-a513-4847-9617-cc7a99feb57e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:24.314080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c0f924-a513-4847-9617-cc7a99feb57e None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:24.321778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 621/1246] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:16:24 2022] GET /identity/v3/role_assignments?user.id=a0eae5ef8c1747408cbaad55be572987&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=54eab69c526d4d53807c7d0f9b1862c7 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:26.455880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c4d86-17c8-4996-adcf-c2b6a7789861 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:26.457000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c4d86-17c8-4996-adcf-c2b6a7789861 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:26.457121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3c4d86-17c8-4996-adcf-c2b6a7789861 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:26.458414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 626/1247] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:26.464817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0f4cb-750b-46ca-8195-f806657a82c9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:26.465425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0f4cb-750b-46ca-8195-f806657a82c9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:26.465425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae0f4cb-750b-46ca-8195-f806657a82c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:26.471233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:26.485199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2ae0f4cb-750b-46ca-8195-f806657a82c9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:26.486782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ae0f4cb-750b-46ca-8195-f806657a82c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:26.567783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 622/1248] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:26.591581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c4d86-17c8-4996-adcf-c2b6a7789861 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:26.612928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:26.712607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3c4d86-17c8-4996-adcf-c2b6a7789861 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:26.751534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:26.752440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:26.752728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:26.753556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:26.766396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:16:26.774192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:26.775496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:26.776581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 627/1249] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:16:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:26.781686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0f4cb-750b-46ca-8195-f806657a82c9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:26.894076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae0f4cb-750b-46ca-8195-f806657a82c9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:26.926822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:26.932643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:26.933347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:26.933666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:26.933945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:26.934757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:26.935067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:26.936651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:26.946961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 623/1250] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:16:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:26.952772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:27.039386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:27.067084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec7afd-9655-4a3a-bbbf-ee658e5da57c None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:27.104042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:27.104841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:27.105127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:27.105402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:27.116740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:27.118185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:27.135374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:27.136487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:27.142604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:27.142604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:27.142604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:27.142604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:27.142604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 628/1251] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:27.142926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:27.250752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e3be7e-92db-41b5-98c9-3f109bc4ab4b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:27.297629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:27.298257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:27.298567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:27.298849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:27.299584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:27.300426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:27.301803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:27.316435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 624/1252] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:27.322276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:27.433273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edc2d96-0830-44bd-afc4-cd44cb265b4d None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:27.465545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:27.474634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:27.475430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:27.475712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:27.476004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:27.483731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:27.485185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:16:27.499567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:27.500981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:27.502007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 629/1253] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:27.507400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:27.569551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:27.615196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac94cf2-13a1-4280-bfb8-39eaee7f6987 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:27.649886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:27.650554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:27.650831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:27.651106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:27.651979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:27.652277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:27.653701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:27.671074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 625/1254] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jul 7 10:16:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:16:27.677511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:27.790191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4836e25-b106-41af-b91d-cd907fd76c01 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:27.825633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:27.826158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:27.826449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:27.826729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:27.883481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:27.897479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:27.899136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:28.009227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 630/1255] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:16:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 07 10:16:28.015322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:28.126987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60895ef3-0bf1-4f10-a5cd-7c6d52e6d682 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:28.142477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:28.164692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:28.165568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:28.165929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:28.166270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:28.176759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:28.178216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:28.198931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:28.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:28.201146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:28.201146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:28.201146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:28.201146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:28.201146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 626/1256] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:28.206185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:28.300520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:28.317361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96973346-bd16-46f7-938b-185ac48f6dc3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:28.354468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:28.355224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:28.355534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:28.355831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:28.356621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:28.356943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:28.358426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:28.371433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 631/1257] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:28.377756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:28.485208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6f62e-752e-4b9a-83b9-a35b648c90f4 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:28.521330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:28.522159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:28.522441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:28.522715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:28.530254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:28.531722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:16:28.546771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:28.547968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:28.548980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 627/1258] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:16:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:28.554656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:28.658003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375887cf-cde2-4584-b238-f95d7f935ab1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:28.693671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f543439-3444-484d-8348-b9a599bd1d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:28.694347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543439-3444-484d-8348-b9a599bd1d24 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:28.694628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543439-3444-484d-8348-b9a599bd1d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:28.694901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f543439-3444-484d-8348-b9a599bd1d24 None admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:28.695689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f543439-3444-484d-8348-b9a599bd1d24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:28.696029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f543439-3444-484d-8348-b9a599bd1d24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:28.697487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f543439-3444-484d-8348-b9a599bd1d24 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:28.715537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 632/1259] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:16:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:28.716824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:28.744360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:28.820842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6e7b4d-4389-45c1-8b7a-6f08e287decb None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:28.854550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17715a35-5d12-4e94-b279-5b72f0cd8634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:28.855209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17715a35-5d12-4e94-b279-5b72f0cd8634 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:28.855492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17715a35-5d12-4e94-b279-5b72f0cd8634 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:28.855800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17715a35-5d12-4e94-b279-5b72f0cd8634 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:28.856637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17715a35-5d12-4e94-b279-5b72f0cd8634 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=a0eae5ef8c1747408cbaad55be572987) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:28.856951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17715a35-5d12-4e94-b279-5b72f0cd8634 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:28.858378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17715a35-5d12-4e94-b279-5b72f0cd8634 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:28.863858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 628/1260] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:16:28 2022] GET /identity/v3/role_assignments?user.id=a0eae5ef8c1747408cbaad55be572987&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=f5019e69971843dfadbbc6be956053fb => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:30.871511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0e0a-5365-4bfe-81ba-fef6151717e1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:30.872084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0e0a-5365-4bfe-81ba-fef6151717e1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:30.872084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffae0e0a-5365-4bfe-81ba-fef6151717e1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:30.873445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 633/1261] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:30.878996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0d46a-3322-4d97-b67d-34d6c5575056 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:30.879364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0d46a-3322-4d97-b67d-34d6c5575056 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:30.879549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b0d46a-3322-4d97-b67d-34d6c5575056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:30.896336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a4b0d46a-3322-4d97-b67d-34d6c5575056 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:30.897507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a4b0d46a-3322-4d97-b67d-34d6c5575056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:30.960888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 629/1262] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:30.982624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae0e0a-5365-4bfe-81ba-fef6151717e1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:30.993255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:31.081816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffae0e0a-5365-4bfe-81ba-fef6151717e1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:31.113839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:31.114604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:31.114882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:31.115151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:31.124854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:16:31.135122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:31.136245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:31.142108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 634/1263] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:16:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:31.142973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0d46a-3322-4d97-b67d-34d6c5575056 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:31.159991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:31.237932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b0d46a-3322-4d97-b67d-34d6c5575056 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:31.273180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:31.273775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:31.274056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:31.274333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:31.275079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:31.275381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:31.276778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:31.285170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 630/1264] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:16:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:31.291147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:31.359370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:31.393110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab6e6a-4a5f-436d-bb5a-aa7317b7b3de None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:31.426015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:31.426791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:31.427075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:31.427351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:31.437874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:31.439301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:31.455995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:31.457042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:31.458178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:31.458178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:31.458178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:31.458178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:31.458178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 635/1265] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:31.462933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:31.548001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:31.567755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0743d0f6-3786-4b7f-b856-83c023690a77 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:31.599645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:31.600325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:31.600608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:31.600887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:31.601597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:31.601904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:31.603261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:31.615256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 631/1266] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:31.621200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:31.717956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032dc9e-fd70-4c73-81ae-696a4dbe4614 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:31.736938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:31.751156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:31.751988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:31.752270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:31.752543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:31.759649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:31.761101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:16:31.775473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:31.781284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:31.782485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 636/1267] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 07 10:16:31.783204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:31.879953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802d962d-3ab4-42ea-8438-2964cfbd2474 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:31.913900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:31.914572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:31.914857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:31.915132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:31.916001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:31.916312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:31.917720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:31.926024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:31.934550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 632/1268] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jul 7 10:16:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:31.940220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:32.036761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c809e-08aa-463c-8500-52ac47c565e1 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:32.068316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:32.069087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:32.069369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:32.069644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:32.079793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:32.081779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:32.098830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:32.099896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:32.101000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:32.101000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:32.101000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:32.101000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:32.101000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 637/1269] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:32.105479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:32.200236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572bc550-ef5e-4545-94f9-cdc5a5b11cad None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:32.232306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:32.232950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:32.233230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:32.233502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:32.234206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:32.234506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:32.235871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:32.246449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 633/1270] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:32.252306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:32.286974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:32.352771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20029cc4-7dd0-452f-9039-90b270c3adfd None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:32.384750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:32.385533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:32.385817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:32.386093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:32.392560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:32.394589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:16:32.408452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:32.409489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:32.410848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 638/1271] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:16:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:32.415405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:32.472599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:32.513289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d1571-895a-47eb-bed9-d723e86f7ed5 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:32.546057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa42c4c-43b2-4496-add3-6797434ca68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:32.546656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa42c4c-43b2-4496-add3-6797434ca68b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:32.546942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa42c4c-43b2-4496-add3-6797434ca68b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:32.547220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa42c4c-43b2-4496-add3-6797434ca68b None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:32.548032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa42c4c-43b2-4496-add3-6797434ca68b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:32.548344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa42c4c-43b2-4496-add3-6797434ca68b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:32.550290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa42c4c-43b2-4496-add3-6797434ca68b None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:32.561992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 634/1272] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:16:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:32.568497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:32.657483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:32.667217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100d9175-1779-44f3-ac80-83c38fa32ea4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:32.699883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabba69d-9a95-463b-97e5-a77e72ca0c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:32.700756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabba69d-9a95-463b-97e5-a77e72ca0c52 None admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:32.701042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabba69d-9a95-463b-97e5-a77e72ca0c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:32.701322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabba69d-9a95-463b-97e5-a77e72ca0c52 None admin] PATH_INFO: `/v3/projects/3d6daaae230949659a794a503d781fb2/users/a0eae5ef8c1747408cbaad55be572987/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:32.703253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabba69d-9a95-463b-97e5-a77e72ca0c52 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d6daaae230949659a794a503d781fb2, user_id=a0eae5ef8c1747408cbaad55be572987, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:32.704707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabba69d-9a95-463b-97e5-a77e72ca0c52 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:32.717962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 639/1273] 10.222.0.16 () {58 vars in 1439 bytes} [Thu Jul 7 10:16:32 2022] PUT /identity/v3/projects/3d6daaae230949659a794a503d781fb2/users/a0eae5ef8c1747408cbaad55be572987/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:16:34.703088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a9a9-3392-42e9-a048-95639c84fff8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:34.703579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a9a9-3392-42e9-a048-95639c84fff8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:34.703693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3a9a9-3392-42e9-a048-95639c84fff8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:34.705041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 635/1274] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:34.710950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76bf37-4e5a-4c7c-87ed-bff338b358f9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:34.711420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76bf37-4e5a-4c7c-87ed-bff338b358f9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:34.711506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76bf37-4e5a-4c7c-87ed-bff338b358f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:34.728095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d76bf37-4e5a-4c7c-87ed-bff338b358f9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:34.729284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d76bf37-4e5a-4c7c-87ed-bff338b358f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:34.799835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 640/1275] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:34.823561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3a9a9-3392-42e9-a048-95639c84fff8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:34.922660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3a9a9-3392-42e9-a048-95639c84fff8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:34.940852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:34.957098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:34.957886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:34.958168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:34.958444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:34.968188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 07 10:16:34.975830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:34.976882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:34.977750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 636/1276] 10.222.0.16 () {56 vars in 1096 bytes} [Thu Jul 7 10:16:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:34.982957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76bf37-4e5a-4c7c-87ed-bff338b358f9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:35.061705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:35.080166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76bf37-4e5a-4c7c-87ed-bff338b358f9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:35.113222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:35.113882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:35.114162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:35.114433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:35.115167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:35.115470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:35.116864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:35.124592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 641/1277] 10.222.0.16 () {56 vars in 1099 bytes} [Thu Jul 7 10:16:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:35.130214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:35.226923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae3613-6fca-44ff-8c56-5ba9d5ab3a8b None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:35.260677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:35.261608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:35.261962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:35.262368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:35.275537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:35.277517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:35.295495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:35.296558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:35.297826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:35.297826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:35.297826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:35.297826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:35.297826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 637/1278] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:35.302414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:35.398562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad833107-a88a-4e74-b01e-bb8fc29c2de4 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:35.429987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:35.430880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:35.431158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:35.431431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:35.432164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:35.432472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:35.433825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:35.445563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 642/1279] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:35.450948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:35.481213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:35.548811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9996f8d-4a05-4c6a-9abd-d009380782e1 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:35.582323 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:35.583120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:35.583404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:35.583682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:35.590920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:35.592391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 07 10:16:35.606371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:35.607282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:35.608408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 638/1280] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:35.613721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:35.617468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:35.712028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a238f5e1-1917-4efa-b5c4-fc6349f2dfd3 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:35.744935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:35.745592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:35.745875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:35.746147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:35.746962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:35.747272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:35.748714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:35.765605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 643/1281] 10.222.0.16 () {56 vars in 1143 bytes} [Thu Jul 7 10:16:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:35.771208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:35.858380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:35.868491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12583a9a-f1df-4719-8e70-2c297c3a9036 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:35.901713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:35.902359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:35.902647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:35.902927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:35.967813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:35.969210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:36.053490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:36.063948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 639/1282] 10.222.0.16 () {58 vars in 1282 bytes} [Thu Jul 7 10:16:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 07 10:16:36.069822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:36.128790 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:36.165696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a3254a-5f90-4a35-bb21-863566c84697 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:36.198424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:36.199196 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:36.199476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:36.199771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:36.209432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:36.210901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 07 10:16:36.228531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:36.229591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:36.230849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 07 10:16:36.230849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 07 10:16:36.230849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 07 10:16:36.230849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:36.230849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 644/1283] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:16:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:36.235160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:36.333842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c643b31d-c86b-4c77-bb41-e48860001576 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:36.367043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:36.367695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:36.368020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:36.368295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:36.369012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:36.369319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:36.370686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:36.381468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 640/1284] 10.222.0.16 () {56 vars in 1109 bytes} [Thu Jul 7 10:16:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:36.387002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:36.484035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd39b6f4-8360-4db8-a1d6-8db7e7e9ac08 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:36.497245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:36.516991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:36.517550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:36.518352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:36.518623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:36.525158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:36.526579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 07 10:16:36.540599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:36.541666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: ERROR keystone.server.flask.application Jul 07 10:16:36.542648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 645/1285] 10.222.0.16 () {56 vars in 1111 bytes} [Thu Jul 7 10:16:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:36.547416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:36.596547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:36.644560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c503c-1e7f-402c-9ec3-907d5edbe342 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:36.677062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff18b17-82fc-49cf-ba57-97887e4dc677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:36.677706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff18b17-82fc-49cf-ba57-97887e4dc677 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:36.677986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff18b17-82fc-49cf-ba57-97887e4dc677 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:36.678257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff18b17-82fc-49cf-ba57-97887e4dc677 None admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:36.679036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff18b17-82fc-49cf-ba57-97887e4dc677 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:36.679343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff18b17-82fc-49cf-ba57-97887e4dc677 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:36.681039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff18b17-82fc-49cf-ba57-97887e4dc677 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:36.692459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 641/1286] 10.222.0.16 () {56 vars in 1148 bytes} [Thu Jul 7 10:16:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:36.698263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:36.793631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1326e06-d09a-4333-9f07-55dfbb1195d7 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:36.825859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb234c72-c68f-41d1-b656-46383ce0cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:36.826492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb234c72-c68f-41d1-b656-46383ce0cf88 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:36.826773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb234c72-c68f-41d1-b656-46383ce0cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:36.827046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb234c72-c68f-41d1-b656-46383ce0cf88 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:36.827896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb234c72-c68f-41d1-b656-46383ce0cf88 None admin] RBAC: Adding query filter params (role.id=f5019e69971843dfadbbc6be956053fb, scope.project.id=3d6daaae230949659a794a503d781fb2, user.id=a0eae5ef8c1747408cbaad55be572987) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:36.828210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb234c72-c68f-41d1-b656-46383ce0cf88 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:36.829619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb234c72-c68f-41d1-b656-46383ce0cf88 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:36.835223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 646/1287] 10.222.0.16 () {56 vars in 1374 bytes} [Thu Jul 7 10:16:36 2022] GET /identity/v3/role_assignments?user.id=a0eae5ef8c1747408cbaad55be572987&scope.project.id=3d6daaae230949659a794a503d781fb2&role.id=f5019e69971843dfadbbc6be956053fb => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:38.838203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21cac-96e9-48cf-8475-0489e7aad435 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:38.838784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21cac-96e9-48cf-8475-0489e7aad435 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:38.838784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df21cac-96e9-48cf-8475-0489e7aad435 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:38.840144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 642/1288] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:38.845395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97817-e9f9-4df6-ab0e-559f5ed16fa0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:38.845791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97817-e9f9-4df6-ab0e-559f5ed16fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:38.845956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f97817-e9f9-4df6-ab0e-559f5ed16fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:38.862383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e2f97817-e9f9-4df6-ab0e-559f5ed16fa0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:38.864099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e2f97817-e9f9-4df6-ab0e-559f5ed16fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:38.893047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:38.930358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 647/1289] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:38.952028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21cac-96e9-48cf-8475-0489e7aad435 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:39.049092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df21cac-96e9-48cf-8475-0489e7aad435 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:39.059472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:39.083435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:39.084244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:39.084527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:39.084801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:39.085441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:39.086791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:39.098575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:39.099822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:39.100939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 643/1290] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:16:38 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:39.105657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f97817-e9f9-4df6-ab0e-559f5ed16fa0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:39.200662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f97817-e9f9-4df6-ab0e-559f5ed16fa0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:39.233152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7980ea-f588-43fb-a0f9-dbd27ef08eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:39.233799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7980ea-f588-43fb-a0f9-dbd27ef08eca None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:39.234083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7980ea-f588-43fb-a0f9-dbd27ef08eca None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:39.234362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7980ea-f588-43fb-a0f9-dbd27ef08eca None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:39.235057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7980ea-f588-43fb-a0f9-dbd27ef08eca None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:39.235364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7980ea-f588-43fb-a0f9-dbd27ef08eca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:39.236756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7980ea-f588-43fb-a0f9-dbd27ef08eca None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:39.241989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 648/1291] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jul 7 10:16:39 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:39.247398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:39.358394 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77815d31-1221-43d2-a9b5-7b65c143177f None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:39.391147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3cc87-b1d2-4aee-bb81-317a4f78dae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:39.391831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3cc87-b1d2-4aee-bb81-317a4f78dae1 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:39.392125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3cc87-b1d2-4aee-bb81-317a4f78dae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:39.392405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3cc87-b1d2-4aee-bb81-317a4f78dae1 None admin] PATH_INFO: `/v3/services` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:39.393106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3cc87-b1d2-4aee-bb81-317a4f78dae1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:39.393448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3cc87-b1d2-4aee-bb81-317a4f78dae1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:39.394825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3cc87-b1d2-4aee-bb81-317a4f78dae1 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:39.399123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 644/1292] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jul 7 10:16:39 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:41.374705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a420-9c34-4be8-af5c-ec0b412f1b2c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:41.375275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a420-9c34-4be8-af5c-ec0b412f1b2c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:41.375275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3865a420-9c34-4be8-af5c-ec0b412f1b2c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:41.376645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 649/1293] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:41.381630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e64570-8237-43ed-8c3d-b87d34499d8e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:41.382022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e64570-8237-43ed-8c3d-b87d34499d8e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:41.382190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e64570-8237-43ed-8c3d-b87d34499d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:41.398446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-95e64570-8237-43ed-8c3d-b87d34499d8e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:41.399635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-95e64570-8237-43ed-8c3d-b87d34499d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:41.429120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:41.465493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 645/1294] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:41.488403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a420-9c34-4be8-af5c-ec0b412f1b2c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:41.518857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:41.587405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3865a420-9c34-4be8-af5c-ec0b412f1b2c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:41.619832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a31fd3-7ff3-47bb-8c80-b6ee4ff653aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:41.620488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a31fd3-7ff3-47bb-8c80-b6ee4ff653aa None admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:41.620770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a31fd3-7ff3-47bb-8c80-b6ee4ff653aa None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:41.621045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a31fd3-7ff3-47bb-8c80-b6ee4ff653aa None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:41.621765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a31fd3-7ff3-47bb-8c80-b6ee4ff653aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:41.623127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a31fd3-7ff3-47bb-8c80-b6ee4ff653aa None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:41.635636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 650/1295] 10.222.0.16 () {60 vars in 1141 bytes} [Thu Jul 7 10:16:41 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:16:43.618253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391268-dd38-4f31-b928-835e138475bb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:43.618760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391268-dd38-4f31-b928-835e138475bb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:43.618858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1391268-dd38-4f31-b928-835e138475bb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:43.620194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 646/1296] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:43.625268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5d0f-7c76-4bd0-8ebc-cde570e8d8f6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:43.625650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5d0f-7c76-4bd0-8ebc-cde570e8d8f6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:43.625822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfc5d0f-7c76-4bd0-8ebc-cde570e8d8f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:43.642599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7bfc5d0f-7c76-4bd0-8ebc-cde570e8d8f6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:43.644439 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7bfc5d0f-7c76-4bd0-8ebc-cde570e8d8f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:43.714492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 651/1297] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:43 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:43.738049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1391268-dd38-4f31-b928-835e138475bb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:43.835972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1391268-dd38-4f31-b928-835e138475bb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:43.868794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:43.869566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:43.869849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:43.870124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:43.870773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:43.872170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:43.883874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:43.885019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:43.885865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 647/1298] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:16:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:43.890959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5d0f-7c76-4bd0-8ebc-cde570e8d8f6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:43.909770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:43.988417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfc5d0f-7c76-4bd0-8ebc-cde570e8d8f6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:44.020792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:44.021441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:44.021722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:44.022003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:44.022692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:44.022995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:44.024395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:44.029394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 652/1299] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jul 7 10:16:43 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:44.035996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:44.066895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:44.132708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f878b-6bef-457b-b230-20b020ad78b0 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:44.164777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b54e77-0ca1-44d8-a95b-7fb54a822c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:44.165429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54e77-0ca1-44d8-a95b-7fb54a822c36 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:44.165714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54e77-0ca1-44d8-a95b-7fb54a822c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:44.165991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b54e77-0ca1-44d8-a95b-7fb54a822c36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:44.166773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54e77-0ca1-44d8-a95b-7fb54a822c36 None admin] RBAC: Adding query filter params (interface=public, service_id=046c475cf5184cbf8c8cfbeaea12db2b, region_id=RegionOne) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:44.167083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54e77-0ca1-44d8-a95b-7fb54a822c36 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:44.168514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b54e77-0ca1-44d8-a95b-7fb54a822c36 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:44.173332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 648/1300] 10.222.0.16 () {56 vars in 1251 bytes} [Thu Jul 7 10:16:44 2022] GET /identity/v3/endpoints?service_id=046c475cf5184cbf8c8cfbeaea12db2b&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:44.179398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:44.258349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:44.277496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bd1b6-2e8d-4636-8280-1cc1a3ec6b11 None admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:44.314094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36bb65-eef1-4c60-b962-3c36c0da3c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:44.314803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36bb65-eef1-4c60-b962-3c36c0da3c7c None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:44.315327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36bb65-eef1-4c60-b962-3c36c0da3c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:44.315645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36bb65-eef1-4c60-b962-3c36c0da3c7c None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:44.316420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36bb65-eef1-4c60-b962-3c36c0da3c7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:44.318246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36bb65-eef1-4c60-b962-3c36c0da3c7c None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:44.325562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 653/1301] 10.222.0.16 () {56 vars in 1087 bytes} [Thu Jul 7 10:16:44 2022] GET /identity/v3/services => generated 2191 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:16:46.307736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e980a6-896b-4981-a393-c017f7baadda None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:46.308303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e980a6-896b-4981-a393-c017f7baadda None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:46.308303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e980a6-896b-4981-a393-c017f7baadda None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:46.309836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 649/1302] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:16:46.314678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d38da2-3dbe-45b6-a65f-98752182d641 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:46.315070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d38da2-3dbe-45b6-a65f-98752182d641 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:46.315270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d38da2-3dbe-45b6-a65f-98752182d641 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:46.332098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e0d38da2-3dbe-45b6-a65f-98752182d641 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:16:46.333278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e0d38da2-3dbe-45b6-a65f-98752182d641 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:16:46.397759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 654/1303] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:16:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 07 10:16:46.420191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e980a6-896b-4981-a393-c017f7baadda None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:46.512137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:46.520011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e980a6-896b-4981-a393-c017f7baadda None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:46.552231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:46.553029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:46.553315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:46.553598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:46.554253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:46.555895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 07 10:16:46.568131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 07 10:16:46.569275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.get_or_create( Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application with Lock( Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self._enter() Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application return self.creator() Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application created_value = creator( Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: ERROR keystone.server.flask.application Jul 07 10:16:46.570433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 650/1304] 10.222.0.16 () {56 vars in 1117 bytes} [Thu Jul 7 10:16:46 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 07 10:16:46.575202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d38da2-3dbe-45b6-a65f-98752182d641 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:46.647494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:46.669681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d38da2-3dbe-45b6-a65f-98752182d641 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:46.702045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ac15a-26f7-4e81-ab64-00312d2d905e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:46.702695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac15a-26f7-4e81-ab64-00312d2d905e None admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:46.702975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac15a-26f7-4e81-ab64-00312d2d905e None admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:46.703248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ac15a-26f7-4e81-ab64-00312d2d905e None admin] PATH_INFO: `/v3/services` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:46.703963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ac15a-26f7-4e81-ab64-00312d2d905e None admin] RBAC: Adding query filter params (name=placement) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:16:46.704266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ac15a-26f7-4e81-ab64-00312d2d905e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:46.706160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ac15a-26f7-4e81-ab64-00312d2d905e None admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:46.711097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 655/1305] 10.222.0.16 () {56 vars in 1116 bytes} [Thu Jul 7 10:16:46 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:16:46.716603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:16:46.812483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96d97b4-358b-4504-bdff-1094e188e1e6 None admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:16:46.844228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141d448b-e2f3-4b27-8983-a3d07585d061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:16:46.844808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d448b-e2f3-4b27-8983-a3d07585d061 None admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:16:46.845089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d448b-e2f3-4b27-8983-a3d07585d061 None admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:16:46.845364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141d448b-e2f3-4b27-8983-a3d07585d061 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:16:46.846139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d448b-e2f3-4b27-8983-a3d07585d061 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:16:46.847480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141d448b-e2f3-4b27-8983-a3d07585d061 None admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:16:46.865208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 651/1306] 10.222.0.16 () {60 vars in 1144 bytes} [Thu Jul 7 10:16:46 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:16:56.708295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=30 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:16:56.860551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:49.067260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78649128-9df6-4fb5-9026-aae36f75b3a6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:49.067865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78649128-9df6-4fb5-9026-aae36f75b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:49.067865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78649128-9df6-4fb5-9026-aae36f75b3a6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:49.072905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 656/1307] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:17:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:17:49.074304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760fefc-cec9-4b61-9ab2-df22c59eb4f2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:49.074743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760fefc-cec9-4b61-9ab2-df22c59eb4f2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:49.074867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760fefc-cec9-4b61-9ab2-df22c59eb4f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:49.165742 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1760fefc-cec9-4b61-9ab2-df22c59eb4f2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:17:49.168304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1760fefc-cec9-4b61-9ab2-df22c59eb4f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:49.363786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 652/1308] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:17:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:17:49.405401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a416c-aaf7-49f4-8743-591da32a6edc None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:49.405691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a416c-aaf7-49f4-8743-591da32a6edc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:49.405984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20a416c-aaf7-49f4-8743-591da32a6edc None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:49.407613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 657/1309] 10.222.0.16 () {54 vars in 964 bytes} [Thu Jul 7 10:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:17:49.423787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eb887-1a81-4c80-939c-ead01f04c51d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:49.424256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eb887-1a81-4c80-939c-ead01f04c51d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:49.424701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173eb887-1a81-4c80-939c-ead01f04c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:49.576034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-173eb887-1a81-4c80-939c-ead01f04c51d None None] MFA Rules not processed for user `61c23b3898c14a99bd1521644345d6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:17:49.577422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-173eb887-1a81-4c80-939c-ead01f04c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:49.605358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:49.671002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 653/1310] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Jul 7 10:17:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:17:49.680751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a416c-aaf7-49f4-8743-591da32a6edc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:17:49.682580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b20a416c-aaf7-49f4-8743-591da32a6edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:49.802781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20a416c-aaf7-49f4-8743-591da32a6edc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:17:49.884156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d791d6ad-4a9f-485f-880b-b2dab6bd591c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:17:49.885116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791d6ad-4a9f-485f-880b-b2dab6bd591c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:49.885576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791d6ad-4a9f-485f-880b-b2dab6bd591c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:49.886042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d791d6ad-4a9f-485f-880b-b2dab6bd591c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:49.888835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d791d6ad-4a9f-485f-880b-b2dab6bd591c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:49.959002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:50.021070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791d6ad-4a9f-485f-880b-b2dab6bd591c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:17:50.023691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d791d6ad-4a9f-485f-880b-b2dab6bd591c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:17:50.045563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 658/1311] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:17:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:17:52.431460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c2d28-ee94-41c8-8072-1a558241e62e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:52.432023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c2d28-ee94-41c8-8072-1a558241e62e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:52.432150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2c2d28-ee94-41c8-8072-1a558241e62e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:52.433439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 654/1312] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:17:52.438702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ffc381-89ad-4d6e-b5c3-5bd12e39d44b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:52.439136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ffc381-89ad-4d6e-b5c3-5bd12e39d44b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:52.439387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ffc381-89ad-4d6e-b5c3-5bd12e39d44b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:52.510935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-75ffc381-89ad-4d6e-b5c3-5bd12e39d44b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:17:52.514827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-75ffc381-89ad-4d6e-b5c3-5bd12e39d44b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:52.616713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 659/1313] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:17:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:17:52.669979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b0435-120c-4121-bb1b-89ad4d3b72b6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:52.670269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b0435-120c-4121-bb1b-89ad4d3b72b6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:52.670550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0b0435-120c-4121-bb1b-89ad4d3b72b6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:52.672041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 655/1314] 10.222.0.16 () {54 vars in 964 bytes} [Thu Jul 7 10:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:17:52.679337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e824-c08f-4a35-8aea-9900d694c278 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:52.679627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e824-c08f-4a35-8aea-9900d694c278 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:52.679945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d2e824-c08f-4a35-8aea-9900d694c278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:52.690345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:52.753831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f1d2e824-c08f-4a35-8aea-9900d694c278 None None] MFA Rules not processed for user `61c23b3898c14a99bd1521644345d6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:17:52.755017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f1d2e824-c08f-4a35-8aea-9900d694c278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:52.832391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 660/1315] 10.222.0.16 () {58 vars in 1061 bytes} [Thu Jul 7 10:17:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:17:52.841638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b0435-120c-4121-bb1b-89ad4d3b72b6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:17:52.843189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ac0b0435-120c-4121-bb1b-89ad4d3b72b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:52.948691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0b0435-120c-4121-bb1b-89ad4d3b72b6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:17:53.021093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:53.023600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d86b26-21bb-4bf5-8a60-f26efde8aa43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:17:53.024546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d86b26-21bb-4bf5-8a60-f26efde8aa43 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:53.025048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d86b26-21bb-4bf5-8a60-f26efde8aa43 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:53.025512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d86b26-21bb-4bf5-8a60-f26efde8aa43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:53.027194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f9d86b26-21bb-4bf5-8a60-f26efde8aa43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:53.132692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d86b26-21bb-4bf5-8a60-f26efde8aa43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:17:53.134118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d86b26-21bb-4bf5-8a60-f26efde8aa43 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:17:53.152499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 656/1316] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:17:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:17:55.686027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9777-cfda-468c-8cb1-ad016bbf63b0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:55.686583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9777-cfda-468c-8cb1-ad016bbf63b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:55.686583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114b9777-cfda-468c-8cb1-ad016bbf63b0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:55.688614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 661/1317] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:17:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:17:55.693794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15f185-081c-461d-ad54-95df8dd13c1e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:55.694204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15f185-081c-461d-ad54-95df8dd13c1e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:55.694346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce15f185-081c-461d-ad54-95df8dd13c1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:55.758353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ce15f185-081c-461d-ad54-95df8dd13c1e None None] MFA Rules not processed for user `646457ebc2974d8dad3b9b7800cf8ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:17:55.759574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ce15f185-081c-461d-ad54-95df8dd13c1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:55.810063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:55.853830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 657/1318] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:17:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:17:55.893355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b9777-cfda-468c-8cb1-ad016bbf63b0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:17:55.894907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-114b9777-cfda-468c-8cb1-ad016bbf63b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:55.995980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114b9777-cfda-468c-8cb1-ad016bbf63b0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:17:55.998682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:56.068650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d81a9ea-0bd3-4ce9-9c47-9e0dda1f3d6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:17:56.069287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81a9ea-0bd3-4ce9-9c47-9e0dda1f3d6d service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:56.069564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81a9ea-0bd3-4ce9-9c47-9e0dda1f3d6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:56.069834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d81a9ea-0bd3-4ce9-9c47-9e0dda1f3d6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:56.071436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4d81a9ea-0bd3-4ce9-9c47-9e0dda1f3d6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:56.176255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81a9ea-0bd3-4ce9-9c47-9e0dda1f3d6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:17:56.177957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d81a9ea-0bd3-4ce9-9c47-9e0dda1f3d6d service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:17:56.195218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 662/1319] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:17:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:17:58.726688 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b0aff-60d1-4b3c-be17-cc5321118758 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:58.727274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b0aff-60d1-4b3c-be17-cc5321118758 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:58.727274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8b0aff-60d1-4b3c-be17-cc5321118758 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:58.728625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 658/1320] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:17:58.733648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b2b2-8af6-408a-a738-cc888660dd69 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:58.734067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b2b2-8af6-408a-a738-cc888660dd69 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:58.734178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c44b2b2-8af6-408a-a738-cc888660dd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:58.773256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:58.798073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0c44b2b2-8af6-408a-a738-cc888660dd69 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:17:58.799263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0c44b2b2-8af6-408a-a738-cc888660dd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:58.891965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 663/1321] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:17:58 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:17:58.929646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b0aff-60d1-4b3c-be17-cc5321118758 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:17:58.931159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4d8b0aff-60d1-4b3c-be17-cc5321118758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:59.030762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8b0aff-60d1-4b3c-be17-cc5321118758 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:17:59.101050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb8017-b399-40fb-960f-3d2730081e0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:17:59.101689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8017-b399-40fb-960f-3d2730081e0d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:17:59.101970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8017-b399-40fb-960f-3d2730081e0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:17:59.102242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb8017-b399-40fb-960f-3d2730081e0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:17:59.103860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-59eb8017-b399-40fb-960f-3d2730081e0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:17:59.111303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:17:59.209137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb8017-b399-40fb-960f-3d2730081e0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:17:59.210571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb8017-b399-40fb-960f-3d2730081e0d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:17:59.227981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 659/1322] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:17:58 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:01.899939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780db3-5081-4957-b39e-3a8ce5605f68 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:01.900473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780db3-5081-4957-b39e-3a8ce5605f68 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:01.900473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa780db3-5081-4957-b39e-3a8ce5605f68 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:01.901886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 664/1323] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:01.907799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4be763-5a22-4f20-b510-1333d5649100 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:01.908188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4be763-5a22-4f20-b510-1333d5649100 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:01.908349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4be763-5a22-4f20-b510-1333d5649100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:01.976346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7b4be763-5a22-4f20-b510-1333d5649100 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:01.977575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7b4be763-5a22-4f20-b510-1333d5649100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:02.040502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:02.082846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 660/1324] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:02.118458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780db3-5081-4957-b39e-3a8ce5605f68 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:02.120023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fa780db3-5081-4957-b39e-3a8ce5605f68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:02.226254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa780db3-5081-4957-b39e-3a8ce5605f68 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:02.238266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:02.302721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab62e7ec-8429-4cd4-a5cd-d3dabc85e162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:02.303363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62e7ec-8429-4cd4-a5cd-d3dabc85e162 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:02.303644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62e7ec-8429-4cd4-a5cd-d3dabc85e162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:02.303942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab62e7ec-8429-4cd4-a5cd-d3dabc85e162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:02.305560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ab62e7ec-8429-4cd4-a5cd-d3dabc85e162 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:02.414860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62e7ec-8429-4cd4-a5cd-d3dabc85e162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:02.416223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab62e7ec-8429-4cd4-a5cd-d3dabc85e162 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:02.434658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 665/1325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:04.507201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c6a2-72aa-4409-aadc-ac9a244e6c36 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:04.507811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c6a2-72aa-4409-aadc-ac9a244e6c36 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:04.507811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99c6a2-72aa-4409-aadc-ac9a244e6c36 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:04.509078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 661/1326] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:04.514906 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344239b9-3788-45ed-9c74-d25f4ddc97b0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:04.514906 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344239b9-3788-45ed-9c74-d25f4ddc97b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:04.516071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344239b9-3788-45ed-9c74-d25f4ddc97b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:04.561347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:04.580174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-344239b9-3788-45ed-9c74-d25f4ddc97b0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:04.581382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-344239b9-3788-45ed-9c74-d25f4ddc97b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:04.676984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 666/1327] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:04.712063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c6a2-72aa-4409-aadc-ac9a244e6c36 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:04.713599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ae99c6a2-72aa-4409-aadc-ac9a244e6c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:04.818397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae99c6a2-72aa-4409-aadc-ac9a244e6c36 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:04.896451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973ef2e3-a2e0-4c3a-80d7-71dd3f30b1a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:04.897113 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef2e3-a2e0-4c3a-80d7-71dd3f30b1a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:04.897398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef2e3-a2e0-4c3a-80d7-71dd3f30b1a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:04.897676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973ef2e3-a2e0-4c3a-80d7-71dd3f30b1a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:04.899309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-973ef2e3-a2e0-4c3a-80d7-71dd3f30b1a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:04.916643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:05.007938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef2e3-a2e0-4c3a-80d7-71dd3f30b1a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:05.009636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973ef2e3-a2e0-4c3a-80d7-71dd3f30b1a9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:05.032051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 662/1328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:07.096005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d62d76-594c-4acb-9d7c-c4eeecaf7e58 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:07.096564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d62d76-594c-4acb-9d7c-c4eeecaf7e58 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:07.096564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d62d76-594c-4acb-9d7c-c4eeecaf7e58 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:07.097932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 667/1329] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:07.103056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405e7d-1b69-48d4-8d36-7d32a0090961 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:07.103462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405e7d-1b69-48d4-8d36-7d32a0090961 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:07.103776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5405e7d-1b69-48d4-8d36-7d32a0090961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:07.170128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e5405e7d-1b69-48d4-8d36-7d32a0090961 None None] MFA Rules not processed for user `646457ebc2974d8dad3b9b7800cf8ac7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:07.171946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e5405e7d-1b69-48d4-8d36-7d32a0090961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:07.237260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:07.273786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 663/1330] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:07 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:07.311010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d62d76-594c-4acb-9d7c-c4eeecaf7e58 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:07.312546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a9d62d76-594c-4acb-9d7c-c4eeecaf7e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:07.414091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d62d76-594c-4acb-9d7c-c4eeecaf7e58 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:07.434516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:07.491971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af186f27-8235-46ad-93d0-6bf16e90a908 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:07.492631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186f27-8235-46ad-93d0-6bf16e90a908 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:07.492915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186f27-8235-46ad-93d0-6bf16e90a908 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:07.493192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af186f27-8235-46ad-93d0-6bf16e90a908 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:07.494816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af186f27-8235-46ad-93d0-6bf16e90a908 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:07.603221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af186f27-8235-46ad-93d0-6bf16e90a908 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:07.604506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af186f27-8235-46ad-93d0-6bf16e90a908 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:07.632579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 668/1331] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:07 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:09.908293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e854b-06ca-4efd-8bb1-add417820cd1 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:09.908859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e854b-06ca-4efd-8bb1-add417820cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:09.908859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e854b-06ca-4efd-8bb1-add417820cd1 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:09.910223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 664/1332] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:09.915234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c93143-6a9a-4686-bfa2-08cf09588e51 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:09.915618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c93143-6a9a-4686-bfa2-08cf09588e51 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:09.915809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c93143-6a9a-4686-bfa2-08cf09588e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:09.978169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:09.981832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-96c93143-6a9a-4686-bfa2-08cf09588e51 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:09.984011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-96c93143-6a9a-4686-bfa2-08cf09588e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:10.078885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 669/1333] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:10.114716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e854b-06ca-4efd-8bb1-add417820cd1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:10.116328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f32e854b-06ca-4efd-8bb1-add417820cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:10.219172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e854b-06ca-4efd-8bb1-add417820cd1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:10.291021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48d55d0-2664-4984-9a5a-b391f95f913a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:10.291664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d55d0-2664-4984-9a5a-b391f95f913a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:10.291975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d55d0-2664-4984-9a5a-b391f95f913a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:10.292253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48d55d0-2664-4984-9a5a-b391f95f913a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:10.293858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f48d55d0-2664-4984-9a5a-b391f95f913a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:10.320179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:10.406365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d55d0-2664-4984-9a5a-b391f95f913a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:10.407832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48d55d0-2664-4984-9a5a-b391f95f913a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:10.425299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 665/1334] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:12.489586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e59641-a706-4e1b-8f39-2fe5029c4949 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:12.490167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e59641-a706-4e1b-8f39-2fe5029c4949 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:12.490167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e59641-a706-4e1b-8f39-2fe5029c4949 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:12.491506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 670/1335] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:12.496643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd31d8-7c3b-49d1-90fb-1742cc4820e0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:12.497038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd31d8-7c3b-49d1-90fb-1742cc4820e0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:12.497205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3cd31d8-7c3b-49d1-90fb-1742cc4820e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:12.561074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c3cd31d8-7c3b-49d1-90fb-1742cc4820e0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:12.562818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c3cd31d8-7c3b-49d1-90fb-1742cc4820e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:12.634920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:12.658030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 666/1336] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:12.697504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e59641-a706-4e1b-8f39-2fe5029c4949 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:12.698997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-75e59641-a706-4e1b-8f39-2fe5029c4949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:12.800135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e59641-a706-4e1b-8f39-2fe5029c4949 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:12.828592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:12.873962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bc9273-f780-4b4c-a5d7-40e4b15309a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:12.874608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc9273-f780-4b4c-a5d7-40e4b15309a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:12.874889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc9273-f780-4b4c-a5d7-40e4b15309a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:12.875164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bc9273-f780-4b4c-a5d7-40e4b15309a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:12.876814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a5bc9273-f780-4b4c-a5d7-40e4b15309a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:12.984259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc9273-f780-4b4c-a5d7-40e4b15309a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:12.985702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bc9273-f780-4b4c-a5d7-40e4b15309a5 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:13.003329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 671/1337] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:15.339106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd5bc6-b902-438c-bccc-d1b43e900a63 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:15.339653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd5bc6-b902-438c-bccc-d1b43e900a63 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:15.339653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd5bc6-b902-438c-bccc-d1b43e900a63 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:15.341200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 667/1338] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:15.346782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38299-c700-46a0-a2e8-2bb78cda8d30 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:15.347174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38299-c700-46a0-a2e8-2bb78cda8d30 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:15.347334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f38299-c700-46a0-a2e8-2bb78cda8d30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:15.411519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e6f38299-c700-46a0-a2e8-2bb78cda8d30 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:15.412742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6f38299-c700-46a0-a2e8-2bb78cda8d30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:15.420783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:15.508252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 672/1339] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:15.544237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd5bc6-b902-438c-bccc-d1b43e900a63 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:15.545803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-93fd5bc6-b902-438c-bccc-d1b43e900a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:15.647178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd5bc6-b902-438c-bccc-d1b43e900a63 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:15.718621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c303e-3b7d-4b0c-9bef-4a77008ac899 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:15.719256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c303e-3b7d-4b0c-9bef-4a77008ac899 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:15.719541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c303e-3b7d-4b0c-9bef-4a77008ac899 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:15.719842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c303e-3b7d-4b0c-9bef-4a77008ac899 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:15.721455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f92c303e-3b7d-4b0c-9bef-4a77008ac899 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:15.756274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:15.827450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c303e-3b7d-4b0c-9bef-4a77008ac899 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:15.828915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c303e-3b7d-4b0c-9bef-4a77008ac899 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:15.847233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 668/1340] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:19.750043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec842f-ee5f-402f-be50-6316b751d2b7 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:19.750543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec842f-ee5f-402f-be50-6316b751d2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:19.750543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec842f-ee5f-402f-be50-6316b751d2b7 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:19.751924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 673/1341] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:19.757123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f1504-b866-4c62-a64f-1c75eb652775 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:19.757526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f1504-b866-4c62-a64f-1c75eb652775 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:19.757805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2f1504-b866-4c62-a64f-1c75eb652775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:19.828880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1a2f1504-b866-4c62-a64f-1c75eb652775 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:19.830090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1a2f1504-b866-4c62-a64f-1c75eb652775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:19.922042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:19.935387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 669/1342] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:19.974200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec842f-ee5f-402f-be50-6316b751d2b7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:19.976424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d9ec842f-ee5f-402f-be50-6316b751d2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:20.070338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec842f-ee5f-402f-be50-6316b751d2b7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:20.073960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e42596-c980-4d4f-84ea-2fc49baf945e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:20.074608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e42596-c980-4d4f-84ea-2fc49baf945e service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:20.074887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e42596-c980-4d4f-84ea-2fc49baf945e service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:20.075160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e42596-c980-4d4f-84ea-2fc49baf945e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:20.084555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e42596-c980-4d4f-84ea-2fc49baf945e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:20.085985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e42596-c980-4d4f-84ea-2fc49baf945e service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:20.095280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 674/1343] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:22.768305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06aedb-5c18-49a6-8c8c-12e392516da5 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:22.768834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06aedb-5c18-49a6-8c8c-12e392516da5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:22.768834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06aedb-5c18-49a6-8c8c-12e392516da5 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:22.770347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 670/1344] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:22.775347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b81608-ed8a-4405-8e6c-52b4b3008904 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:22.775822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b81608-ed8a-4405-8e6c-52b4b3008904 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:22.776006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b81608-ed8a-4405-8e6c-52b4b3008904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:22.804521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c1b81608-ed8a-4405-8e6c-52b4b3008904 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:22.805726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1b81608-ed8a-4405-8e6c-52b4b3008904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:22.834826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:22.887458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 675/1345] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:22.924955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06aedb-5c18-49a6-8c8c-12e392516da5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:22.936828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06aedb-5c18-49a6-8c8c-12e392516da5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:22.940469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450520a-484b-458b-a269-8ea7a157cbab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:22.941201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450520a-484b-458b-a269-8ea7a157cbab service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:22.941555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450520a-484b-458b-a269-8ea7a157cbab service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:22.941900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450520a-484b-458b-a269-8ea7a157cbab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:22.952875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450520a-484b-458b-a269-8ea7a157cbab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:22.954346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450520a-484b-458b-a269-8ea7a157cbab service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:22.964927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 671/1346] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:26.937433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45432122-12ea-4e5b-8b9f-0239b7a9f2db None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:26.937989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45432122-12ea-4e5b-8b9f-0239b7a9f2db None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:26.937989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45432122-12ea-4e5b-8b9f-0239b7a9f2db None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:26.939494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 676/1347] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:26.944558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e131-340e-49c6-ad09-eb8575cbaa10 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:26.944973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e131-340e-49c6-ad09-eb8575cbaa10 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:26.945245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3652e131-340e-49c6-ad09-eb8575cbaa10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:26.963548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3652e131-340e-49c6-ad09-eb8575cbaa10 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:26.964798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3652e131-340e-49c6-ad09-eb8575cbaa10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:26.978720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 672/1348] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:27.017551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45432122-12ea-4e5b-8b9f-0239b7a9f2db None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:27.028144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45432122-12ea-4e5b-8b9f-0239b7a9f2db None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:27.032482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fea608a-12ce-452f-bdca-0edc9d749b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:27.033115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea608a-12ce-452f-bdca-0edc9d749b9c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:27.033392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea608a-12ce-452f-bdca-0edc9d749b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:27.033666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fea608a-12ce-452f-bdca-0edc9d749b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:27.042812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea608a-12ce-452f-bdca-0edc9d749b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:27.043940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fea608a-12ce-452f-bdca-0edc9d749b9c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:27.056151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 677/1349] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:32.067568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88302-2c85-4feb-a098-82a7118e4f32 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:32.068040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88302-2c85-4feb-a098-82a7118e4f32 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:32.068151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c88302-2c85-4feb-a098-82a7118e4f32 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:32.069809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 673/1350] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:18:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:32.076001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c35d7-e71b-43fb-81cb-504301c1a2c7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:32.077009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c35d7-e71b-43fb-81cb-504301c1a2c7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:32.077498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c35d7-e71b-43fb-81cb-504301c1a2c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:32.190868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a44c35d7-e71b-43fb-81cb-504301c1a2c7 None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:32.192459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a44c35d7-e71b-43fb-81cb-504301c1a2c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:32.287831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 678/1351] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:18:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:36.533353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7017b23-43c8-4f51-b607-9284a6a883b6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:36.533842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7017b23-43c8-4f51-b607-9284a6a883b6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:36.534023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7017b23-43c8-4f51-b607-9284a6a883b6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:36.535288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 674/1352] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:18:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:36.540411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f863cb-d369-4f91-8f84-b174df587c4e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:36.541356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f863cb-d369-4f91-8f84-b174df587c4e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:36.541730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f863cb-d369-4f91-8f84-b174df587c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:36.560684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-96f863cb-d369-4f91-8f84-b174df587c4e None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:36.561953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-96f863cb-d369-4f91-8f84-b174df587c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:36.570457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 679/1353] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:18:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:36.958356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=13 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:38.158967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb001f0-7058-48e4-b392-6cf9d3630515 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:38.159632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb001f0-7058-48e4-b392-6cf9d3630515 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:38.159632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb001f0-7058-48e4-b392-6cf9d3630515 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:38.160947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 675/1354] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:18:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:38.165838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f57028-be9e-43a5-8dd6-922ccbc9e474 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:38.166288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f57028-be9e-43a5-8dd6-922ccbc9e474 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:38.166388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f57028-be9e-43a5-8dd6-922ccbc9e474 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:38.186004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-49f57028-be9e-43a5-8dd6-922ccbc9e474 None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:38.187185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-49f57028-be9e-43a5-8dd6-922ccbc9e474 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:38.195612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 680/1355] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:18:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:41.629659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbc14e-1fb0-476f-b7eb-fb5b99bd79cc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:41.630680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbc14e-1fb0-476f-b7eb-fb5b99bd79cc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:41.631195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfbc14e-1fb0-476f-b7eb-fb5b99bd79cc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:41.633299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 676/1356] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:41.638981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbdb4b-7023-40b2-85f2-b09819b73126 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:41.640355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbdb4b-7023-40b2-85f2-b09819b73126 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:41.640638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcbdb4b-7023-40b2-85f2-b09819b73126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:41.658796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2bcbdb4b-7023-40b2-85f2-b09819b73126 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:41.660016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2bcbdb4b-7023-40b2-85f2-b09819b73126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:41.668124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 681/1357] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:41.730254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a14a69-85b8-4078-9121-a90aa56ecbfd None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:41.731410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a14a69-85b8-4078-9121-a90aa56ecbfd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:41.732507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a14a69-85b8-4078-9121-a90aa56ecbfd None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:41.734953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 677/1358] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jul 7 10:18:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:41.744363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa756af-e33b-4a4f-8216-d8e93b89e37c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:41.744955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa756af-e33b-4a4f-8216-d8e93b89e37c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:41.745551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa756af-e33b-4a4f-8216-d8e93b89e37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:41.788639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:41.798588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6aa756af-e33b-4a4f-8216-d8e93b89e37c None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:41.800724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6aa756af-e33b-4a4f-8216-d8e93b89e37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:41.889518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 682/1359] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jul 7 10:18:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:41.897718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a14a69-85b8-4078-9121-a90aa56ecbfd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:41.913049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a14a69-85b8-4078-9121-a90aa56ecbfd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:41.919207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507f84e-3984-4d3e-8c47-bfa1bf688e69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:41.920104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f84e-3984-4d3e-8c47-bfa1bf688e69 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:41.920391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f84e-3984-4d3e-8c47-bfa1bf688e69 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:41.920670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507f84e-3984-4d3e-8c47-bfa1bf688e69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:41.929611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507f84e-3984-4d3e-8c47-bfa1bf688e69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:41.931181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507f84e-3984-4d3e-8c47-bfa1bf688e69 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:41.944932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 678/1360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:44.169020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adcbe2-e2ed-4b93-afe2-5ca2f00198e4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:44.169565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adcbe2-e2ed-4b93-afe2-5ca2f00198e4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:44.169565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98adcbe2-e2ed-4b93-afe2-5ca2f00198e4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:44.171078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 683/1361] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:44.176321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39db59f-e68b-480d-8713-4371d96a6513 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:44.176719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39db59f-e68b-480d-8713-4371d96a6513 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:44.176882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39db59f-e68b-480d-8713-4371d96a6513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:44.195002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a39db59f-e68b-480d-8713-4371d96a6513 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:44.196506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a39db59f-e68b-480d-8713-4371d96a6513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:44.205088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 679/1362] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:44.248253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adcbe2-e2ed-4b93-afe2-5ca2f00198e4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:44.263639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98adcbe2-e2ed-4b93-afe2-5ca2f00198e4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:44.270378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d77ba-f6ef-4d58-8911-80ef06cd59e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:44.271428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d77ba-f6ef-4d58-8911-80ef06cd59e6 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:44.271875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d77ba-f6ef-4d58-8911-80ef06cd59e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:44.272252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d77ba-f6ef-4d58-8911-80ef06cd59e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:44.283660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d77ba-f6ef-4d58-8911-80ef06cd59e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:44.285277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d77ba-f6ef-4d58-8911-80ef06cd59e6 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:44.295326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 684/1363] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:46.631550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f61992-330a-4c10-ad03-e5e9c8433180 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:46.632131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f61992-330a-4c10-ad03-e5e9c8433180 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:46.632273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f61992-330a-4c10-ad03-e5e9c8433180 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:46.633539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 680/1364] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:46.639182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502ff0-691c-44b3-b622-1175c8080c21 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:46.639665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502ff0-691c-44b3-b622-1175c8080c21 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:46.639791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7502ff0-691c-44b3-b622-1175c8080c21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:46.659130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e7502ff0-691c-44b3-b622-1175c8080c21 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:46.660647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e7502ff0-691c-44b3-b622-1175c8080c21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:46.671553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 685/1365] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:46.714382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eea397-5b5a-4fb1-b505-6fb3ab392f14 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:46.714701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eea397-5b5a-4fb1-b505-6fb3ab392f14 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:46.715579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eea397-5b5a-4fb1-b505-6fb3ab392f14 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:46.716964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 681/1366] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jul 7 10:18:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:46.722252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02eb62-a35d-4c2e-b259-88dabe7cfeaa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:46.722534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02eb62-a35d-4c2e-b259-88dabe7cfeaa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:46.722804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b02eb62-a35d-4c2e-b259-88dabe7cfeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:46.740747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6b02eb62-a35d-4c2e-b259-88dabe7cfeaa None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:46.741927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6b02eb62-a35d-4c2e-b259-88dabe7cfeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:46.751158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 686/1367] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jul 7 10:18:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:18:46.758974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eea397-5b5a-4fb1-b505-6fb3ab392f14 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:46.773372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eea397-5b5a-4fb1-b505-6fb3ab392f14 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:46.779167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf354f46-4e2b-4a9a-9770-223b87193366 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:46.779821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf354f46-4e2b-4a9a-9770-223b87193366 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:46.780106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf354f46-4e2b-4a9a-9770-223b87193366 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:46.780383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf354f46-4e2b-4a9a-9770-223b87193366 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:46.788235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf354f46-4e2b-4a9a-9770-223b87193366 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:46.789802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf354f46-4e2b-4a9a-9770-223b87193366 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:46.799170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 682/1368] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:48.865603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb73a4-284a-4c16-9c36-9edfbd805f4d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:48.866183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb73a4-284a-4c16-9c36-9edfbd805f4d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:48.866183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb73a4-284a-4c16-9c36-9edfbd805f4d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:48.867589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 687/1369] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:48.873351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ccd74-48fa-40e9-8e62-72c474ec727e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:48.873819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ccd74-48fa-40e9-8e62-72c474ec727e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:48.873899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916ccd74-48fa-40e9-8e62-72c474ec727e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:48.893161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-916ccd74-48fa-40e9-8e62-72c474ec727e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:48.894533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-916ccd74-48fa-40e9-8e62-72c474ec727e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:48.902920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 683/1370] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:48.949802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb73a4-284a-4c16-9c36-9edfbd805f4d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:48.964278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb73a4-284a-4c16-9c36-9edfbd805f4d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:48.970180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51b73c-c564-49d7-8ba9-415d27662733 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:48.970979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51b73c-c564-49d7-8ba9-415d27662733 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:48.971930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51b73c-c564-49d7-8ba9-415d27662733 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:48.971930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51b73c-c564-49d7-8ba9-415d27662733 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:48.981532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51b73c-c564-49d7-8ba9-415d27662733 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:48.983217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51b73c-c564-49d7-8ba9-415d27662733 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:48.993179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 688/1371] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:49.677905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d923b4-fc5e-45c4-a145-beaf1951bf3b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:49.678351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d923b4-fc5e-45c4-a145-beaf1951bf3b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:49.678736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d923b4-fc5e-45c4-a145-beaf1951bf3b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:49.680281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 684/1372] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:49.685488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f091d-f3f6-48b0-b321-96a5407ee44e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:49.685779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f091d-f3f6-48b0-b321-96a5407ee44e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:49.686056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2f091d-f3f6-48b0-b321-96a5407ee44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:49.704358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8c2f091d-f3f6-48b0-b321-96a5407ee44e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:49.705547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8c2f091d-f3f6-48b0-b321-96a5407ee44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:49.713561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 689/1373] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:50.052270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ded9b3-65b9-4c3d-9fec-3d473bc00f2e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:50.052270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ded9b3-65b9-4c3d-9fec-3d473bc00f2e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:50.052270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ded9b3-65b9-4c3d-9fec-3d473bc00f2e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:50.054662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 685/1374] 10.222.0.16 () {54 vars in 963 bytes} [Thu Jul 7 10:18:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:50.058789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189fa16-a6cf-4b77-acd0-a0872ed36bb4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:50.059254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189fa16-a6cf-4b77-acd0-a0872ed36bb4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:50.059409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f189fa16-a6cf-4b77-acd0-a0872ed36bb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:50.108907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f189fa16-a6cf-4b77-acd0-a0872ed36bb4 None None] MFA Rules not processed for user `cc73785ed9da4ad297b924632b14e92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:50.110098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f189fa16-a6cf-4b77-acd0-a0872ed36bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:50.176514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 690/1375] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Jul 7 10:18:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:50.184335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ded9b3-65b9-4c3d-9fec-3d473bc00f2e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:50.195011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ded9b3-65b9-4c3d-9fec-3d473bc00f2e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:50.198551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f2fea-5b50-4229-9f87-9d0f1968237a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:50.199789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2fea-5b50-4229-9f87-9d0f1968237a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:50.200075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2fea-5b50-4229-9f87-9d0f1968237a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:50.200348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f2fea-5b50-4229-9f87-9d0f1968237a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:50.211007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f2fea-5b50-4229-9f87-9d0f1968237a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:50.211434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f2fea-5b50-4229-9f87-9d0f1968237a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:50.221323 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 686/1376] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:51.054522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc627f-4456-4cb5-b678-89bd69cafd75 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:51.054886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc627f-4456-4cb5-b678-89bd69cafd75 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:51.055102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfc627f-4456-4cb5-b678-89bd69cafd75 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:51.056513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 691/1377] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:51.061739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f7593-488b-469a-ae7f-d665db557801 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:51.062189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f7593-488b-469a-ae7f-d665db557801 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:51.062305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f7593-488b-469a-ae7f-d665db557801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:51.083056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b36f7593-488b-469a-ae7f-d665db557801 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:51.084502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b36f7593-488b-469a-ae7f-d665db557801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:51.092943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 687/1378] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:51.135580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc627f-4456-4cb5-b678-89bd69cafd75 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:51.150009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfc627f-4456-4cb5-b678-89bd69cafd75 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:51.156142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e10989-90b6-4774-9d4e-5e1a60e249fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:51.156721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e10989-90b6-4774-9d4e-5e1a60e249fc service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:51.157004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e10989-90b6-4774-9d4e-5e1a60e249fc service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:51.157280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e10989-90b6-4774-9d4e-5e1a60e249fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:51.165362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e10989-90b6-4774-9d4e-5e1a60e249fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:51.166946 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e10989-90b6-4774-9d4e-5e1a60e249fc service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:51.171444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:18:51.178447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 692/1379] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:53.228093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a654c-176c-4819-baee-6a9d357bfc93 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:53.229028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a654c-176c-4819-baee-6a9d357bfc93 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:53.229512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a654c-176c-4819-baee-6a9d357bfc93 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:53.231478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 688/1380] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:53.237235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce0746-c94f-403b-a95b-2c3504bc9eb3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:53.238152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce0746-c94f-403b-a95b-2c3504bc9eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:53.238654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ce0746-c94f-403b-a95b-2c3504bc9eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:53.257517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e5ce0746-c94f-403b-a95b-2c3504bc9eb3 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:53.258656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e5ce0746-c94f-403b-a95b-2c3504bc9eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:53.268704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 693/1381] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:53.312658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a654c-176c-4819-baee-6a9d357bfc93 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:53.327397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a654c-176c-4819-baee-6a9d357bfc93 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:53.332838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2509d08-4026-4e55-8a5a-89a2124fdced service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:53.333469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2509d08-4026-4e55-8a5a-89a2124fdced service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:53.333751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2509d08-4026-4e55-8a5a-89a2124fdced service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:53.334026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2509d08-4026-4e55-8a5a-89a2124fdced service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:53.342082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2509d08-4026-4e55-8a5a-89a2124fdced service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:53.343641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2509d08-4026-4e55-8a5a-89a2124fdced service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:53.352798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 689/1382] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:55.412364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a180b-2990-498e-a81a-53225fe285af None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:55.412835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a180b-2990-498e-a81a-53225fe285af None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:55.412953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a180b-2990-498e-a81a-53225fe285af None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:55.414275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 694/1383] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:55.419941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa695d1-67fa-47bc-96b3-fd0282adaa89 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:55.420363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa695d1-67fa-47bc-96b3-fd0282adaa89 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:55.420502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa695d1-67fa-47bc-96b3-fd0282adaa89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:55.439371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bfa695d1-67fa-47bc-96b3-fd0282adaa89 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:55.440574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bfa695d1-67fa-47bc-96b3-fd0282adaa89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:55.448769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 690/1384] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:18:55.490074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a180b-2990-498e-a81a-53225fe285af None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:55.505579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a180b-2990-498e-a81a-53225fe285af None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:55.516765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ca17cb-b69f-4af4-bf97-1beddb4d4936 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:55.518248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca17cb-b69f-4af4-bf97-1beddb4d4936 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:55.518694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca17cb-b69f-4af4-bf97-1beddb4d4936 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:55.518973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ca17cb-b69f-4af4-bf97-1beddb4d4936 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:55.528725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca17cb-b69f-4af4-bf97-1beddb4d4936 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:55.530330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ca17cb-b69f-4af4-bf97-1beddb4d4936 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:55.540173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 695/1385] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:18:57.579065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecfc8c-cf29-4dfa-ae4a-7718036ee05c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:57.579980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecfc8c-cf29-4dfa-ae4a-7718036ee05c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:57.579980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fecfc8c-cf29-4dfa-ae4a-7718036ee05c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:57.584716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 691/1386] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:18:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:18:57.585918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cdf4d-f472-41c1-84bf-99c8b4252398 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:57.586460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cdf4d-f472-41c1-84bf-99c8b4252398 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:57.586460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cdf4d-f472-41c1-84bf-99c8b4252398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:57.614283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a69cdf4d-f472-41c1-84bf-99c8b4252398 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:18:57.616219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a69cdf4d-f472-41c1-84bf-99c8b4252398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:18:57.629519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 696/1387] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:18:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:18:57.672807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecfc8c-cf29-4dfa-ae4a-7718036ee05c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:18:57.688159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fecfc8c-cf29-4dfa-ae4a-7718036ee05c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:18:57.693767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b68337-32e5-4268-bf2e-5454c6bf2ebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:18:57.694410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b68337-32e5-4268-bf2e-5454c6bf2ebc service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:18:57.694691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b68337-32e5-4268-bf2e-5454c6bf2ebc service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:18:57.694964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b68337-32e5-4268-bf2e-5454c6bf2ebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:18:57.703587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b68337-32e5-4268-bf2e-5454c6bf2ebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:18:57.705330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b68337-32e5-4268-bf2e-5454c6bf2ebc service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:18:57.716401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 692/1388] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:18:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:19:00.189795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73df00-c5ae-4982-a689-d112f3ded349 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:00.190461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73df00-c5ae-4982-a689-d112f3ded349 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:00.190461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa73df00-c5ae-4982-a689-d112f3ded349 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:00.191780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 697/1389] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:19:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:00.199264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89fec3-70c3-4267-9fd1-59d86305ee07 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:00.199731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89fec3-70c3-4267-9fd1-59d86305ee07 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:00.199972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a89fec3-70c3-4267-9fd1-59d86305ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:00.221411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0a89fec3-70c3-4267-9fd1-59d86305ee07 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:00.222880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0a89fec3-70c3-4267-9fd1-59d86305ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:00.232177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 693/1390] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:19:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:19:00.285933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73df00-c5ae-4982-a689-d112f3ded349 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:00.304206 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa73df00-c5ae-4982-a689-d112f3ded349 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:00.311178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f05044-d803-48eb-b0d3-1ca3d74b22f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:00.312070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f05044-d803-48eb-b0d3-1ca3d74b22f5 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:00.312605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f05044-d803-48eb-b0d3-1ca3d74b22f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:00.312962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f05044-d803-48eb-b0d3-1ca3d74b22f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:00.324277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f05044-d803-48eb-b0d3-1ca3d74b22f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:00.326275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f05044-d803-48eb-b0d3-1ca3d74b22f5 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:00.338266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 698/1391] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:02.382634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611767d-3290-4cf2-a3d2-4186b6ca768a None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:02.383203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611767d-3290-4cf2-a3d2-4186b6ca768a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:02.383203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611767d-3290-4cf2-a3d2-4186b6ca768a None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:02.384638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 694/1392] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:02.390833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c49f3-49f1-4359-9082-af5f15da2c92 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:02.391272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c49f3-49f1-4359-9082-af5f15da2c92 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:02.391616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c49f3-49f1-4359-9082-af5f15da2c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:02.413937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-410c49f3-49f1-4359-9082-af5f15da2c92 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:02.415637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-410c49f3-49f1-4359-9082-af5f15da2c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:02.426495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 699/1393] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:19:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:02.470460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611767d-3290-4cf2-a3d2-4186b6ca768a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:02.485801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611767d-3290-4cf2-a3d2-4186b6ca768a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:02.491401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c706e467-f371-4688-bb70-b007ed04fdb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:02.492080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706e467-f371-4688-bb70-b007ed04fdb3 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:02.492372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706e467-f371-4688-bb70-b007ed04fdb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:02.492656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c706e467-f371-4688-bb70-b007ed04fdb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:02.502177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706e467-f371-4688-bb70-b007ed04fdb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:02.503899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c706e467-f371-4688-bb70-b007ed04fdb3 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:02.515654 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 695/1394] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:04.563078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fbef3-a3ef-4ce8-ae1d-4408bd85b121 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:04.563599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fbef3-a3ef-4ce8-ae1d-4408bd85b121 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:04.563599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990fbef3-a3ef-4ce8-ae1d-4408bd85b121 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:04.565003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 700/1395] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:04.570190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242bb8-366d-4e01-b0f4-df1107227f39 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:04.570575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242bb8-366d-4e01-b0f4-df1107227f39 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:04.570744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1242bb8-366d-4e01-b0f4-df1107227f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:04.589003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b1242bb8-366d-4e01-b0f4-df1107227f39 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:04.590236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1242bb8-366d-4e01-b0f4-df1107227f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:04.598450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 696/1396] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:19:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:04.645617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fbef3-a3ef-4ce8-ae1d-4408bd85b121 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:04.659811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990fbef3-a3ef-4ce8-ae1d-4408bd85b121 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:04.665161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9d44af-27a5-4a9e-a5d7-cb269085089e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:04.665853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d44af-27a5-4a9e-a5d7-cb269085089e service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:04.666141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d44af-27a5-4a9e-a5d7-cb269085089e service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:04.666444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9d44af-27a5-4a9e-a5d7-cb269085089e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:04.675459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d44af-27a5-4a9e-a5d7-cb269085089e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:04.677070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9d44af-27a5-4a9e-a5d7-cb269085089e service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:04.687188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 701/1397] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:05.501820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b362-cffd-4da9-a983-33a18f066b51 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:05.502097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b362-cffd-4da9-a983-33a18f066b51 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:05.502594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c54b362-cffd-4da9-a983-33a18f066b51 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:05.504305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 697/1398] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:19:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:05.509859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbb734-1d19-442a-a72b-225b92eab20f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:05.510149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbb734-1d19-442a-a72b-225b92eab20f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:05.510967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bbb734-1d19-442a-a72b-225b92eab20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:05.529183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-27bbb734-1d19-442a-a72b-225b92eab20f None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:05.530376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-27bbb734-1d19-442a-a72b-225b92eab20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:05.538256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 702/1399] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:19:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:05.592860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af33eed-968c-4606-a89b-b13f2c871f93 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:05.593268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af33eed-968c-4606-a89b-b13f2c871f93 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:05.593433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af33eed-968c-4606-a89b-b13f2c871f93 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:05.594760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 698/1400] 10.222.0.16 () {54 vars in 962 bytes} [Thu Jul 7 10:19:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:05.599560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932268e5-4f49-4019-b7a6-a19964008ffd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:05.599985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932268e5-4f49-4019-b7a6-a19964008ffd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:05.600146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932268e5-4f49-4019-b7a6-a19964008ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:05.647685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-932268e5-4f49-4019-b7a6-a19964008ffd None None] MFA Rules not processed for user `4b320f5e6fc4489484fe6599f4391245`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:05.648897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-932268e5-4f49-4019-b7a6-a19964008ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:05.708229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 703/1401] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Jul 7 10:19:05 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:05.715865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af33eed-968c-4606-a89b-b13f2c871f93 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:05.726363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af33eed-968c-4606-a89b-b13f2c871f93 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:05.730355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bc0aef-3102-40cd-b3b0-0d4d0e4e88d1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:05.730986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0aef-3102-40cd-b3b0-0d4d0e4e88d1 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:05.731269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0aef-3102-40cd-b3b0-0d4d0e4e88d1 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:05.731543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bc0aef-3102-40cd-b3b0-0d4d0e4e88d1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:05.740385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc0aef-3102-40cd-b3b0-0d4d0e4e88d1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:05.741793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bc0aef-3102-40cd-b3b0-0d4d0e4e88d1 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:05.751763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 699/1402] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:19:06.475362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a3fb8-97ed-491e-bfdd-f2366563211d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:06.475768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a3fb8-97ed-491e-bfdd-f2366563211d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:06.476123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a3fb8-97ed-491e-bfdd-f2366563211d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:06.524694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7f3a3fb8-97ed-491e-bfdd-f2366563211d None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:06.526524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7f3a3fb8-97ed-491e-bfdd-f2366563211d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:06.567105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:19:06.592438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 704/1403] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:06.638276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749b313-e051-4558-b40a-eedc17e63e4c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:06.638775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749b313-e051-4558-b40a-eedc17e63e4c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:06.638872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e749b313-e051-4558-b40a-eedc17e63e4c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:06.640282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 700/1404] 10.222.0.16 () {54 vars in 942 bytes} [Thu Jul 7 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:06.646561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877244c-6828-4367-abea-d7ce2432f5a4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:06.646964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877244c-6828-4367-abea-d7ce2432f5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:06.647267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4877244c-6828-4367-abea-d7ce2432f5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:06.711700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4877244c-6828-4367-abea-d7ce2432f5a4 None None] MFA Rules not processed for user `3e2e080fd7144086ac2cf5628a7a68ed`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:06.715701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4877244c-6828-4367-abea-d7ce2432f5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:06.745964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43254152-ac21-48e3-9322-f3a1d016e7f3 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:06.746253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43254152-ac21-48e3-9322-f3a1d016e7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:06.746530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43254152-ac21-48e3-9322-f3a1d016e7f3 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:06.747911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 701/1405] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:19:06.752558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833df5a-3b3c-4100-a441-816ff783a1ed None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:06.752844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833df5a-3b3c-4100-a441-816ff783a1ed None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:06.753123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6833df5a-3b3c-4100-a441-816ff783a1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:06.771722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6833df5a-3b3c-4100-a441-816ff783a1ed None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:06.772969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6833df5a-3b3c-4100-a441-816ff783a1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:06.780852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 702/1406] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:19:06.793397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 705/1407] 10.222.0.16 () {58 vars in 1039 bytes} [Thu Jul 7 10:19:06 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:06.802971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833df5a-3b3c-4100-a441-816ff783a1ed None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:06.815310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6833df5a-3b3c-4100-a441-816ff783a1ed None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:06.820448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64811b22-cc5a-4b45-bf71-abf20f05bd3d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:06.821102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811b22-cc5a-4b45-bf71-abf20f05bd3d service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:06.821390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811b22-cc5a-4b45-bf71-abf20f05bd3d service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:06.821674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64811b22-cc5a-4b45-bf71-abf20f05bd3d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:06.825972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877244c-6828-4367-abea-d7ce2432f5a4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:06.831026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64811b22-cc5a-4b45-bf71-abf20f05bd3d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:06.832747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64811b22-cc5a-4b45-bf71-abf20f05bd3d service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:06.837551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4877244c-6828-4367-abea-d7ce2432f5a4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:06.841472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 703/1408] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:19:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:19:06.843303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ffa95d-a749-4520-93cf-267bcac1ca67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:06.843873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa95d-a749-4520-93cf-267bcac1ca67 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:06.844160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa95d-a749-4520-93cf-267bcac1ca67 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:06.844438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ffa95d-a749-4520-93cf-267bcac1ca67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:06.855414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffa95d-a749-4520-93cf-267bcac1ca67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:06.856917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ffa95d-a749-4520-93cf-267bcac1ca67 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:06.867064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 706/1409] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:16.837701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=69 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:19:16.862627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=59 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:19:28.817234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b98b-82c8-486c-9571-93cdf6b1cab9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:28.817778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b98b-82c8-486c-9571-93cdf6b1cab9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:28.817778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e958b98b-82c8-486c-9571-93cdf6b1cab9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:28.819159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 704/1410] 10.222.0.16 () {54 vars in 918 bytes} [Thu Jul 7 10:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:28.824349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d405c-f7cf-4f79-9787-cc129a12d3ad None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:28.824757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d405c-f7cf-4f79-9787-cc129a12d3ad None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:28.824899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083d405c-f7cf-4f79-9787-cc129a12d3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:28.843305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-083d405c-f7cf-4f79-9787-cc129a12d3ad None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:28.844520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-083d405c-f7cf-4f79-9787-cc129a12d3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:28.852764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 707/1411] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jul 7 10:19:28 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:28.866920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e893f-7f1c-4232-b3ca-6aca2e669dcc None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:28.867215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e893f-7f1c-4232-b3ca-6aca2e669dcc None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:28.867500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e893f-7f1c-4232-b3ca-6aca2e669dcc None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:28.868907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 705/1412] 10.222.0.16 () {54 vars in 918 bytes} [Thu Jul 7 10:19:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:28.873852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bdffa-d49a-44d3-841f-2c90c5020c28 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:28.874140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bdffa-d49a-44d3-841f-2c90c5020c28 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:28.874416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2bdffa-d49a-44d3-841f-2c90c5020c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:28.891564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0c2bdffa-d49a-44d3-841f-2c90c5020c28 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:28.892766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0c2bdffa-d49a-44d3-841f-2c90c5020c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:28.901956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 708/1413] 10.222.0.16 () {58 vars in 1015 bytes} [Thu Jul 7 10:19:28 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:31.363929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135b764-94b0-4a3d-bed6-2fbc0efa552e None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:31.364492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135b764-94b0-4a3d-bed6-2fbc0efa552e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:31.364492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135b764-94b0-4a3d-bed6-2fbc0efa552e None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:31.365852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 706/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:31.371918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64a2d-b19e-4b7f-8bc4-7e46b5f7330e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:31.372321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64a2d-b19e-4b7f-8bc4-7e46b5f7330e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:31.372509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac64a2d-b19e-4b7f-8bc4-7e46b5f7330e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:31.390884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fac64a2d-b19e-4b7f-8bc4-7e46b5f7330e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:31.392100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fac64a2d-b19e-4b7f-8bc4-7e46b5f7330e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:31.400858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 709/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:31.443275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135b764-94b0-4a3d-bed6-2fbc0efa552e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:31.457921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135b764-94b0-4a3d-bed6-2fbc0efa552e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:31.463271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32f3de-f6b2-478b-bd7e-4ff83d76c824 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:31.463937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32f3de-f6b2-478b-bd7e-4ff83d76c824 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:31.464220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32f3de-f6b2-478b-bd7e-4ff83d76c824 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:31.464494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32f3de-f6b2-478b-bd7e-4ff83d76c824 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:31.472943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32f3de-f6b2-478b-bd7e-4ff83d76c824 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:31.474497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32f3de-f6b2-478b-bd7e-4ff83d76c824 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:31.483621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 707/1416] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:32.374858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240ed73-6f6c-461e-b45f-99f12074fef0 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:32.375450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240ed73-6f6c-461e-b45f-99f12074fef0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:32.375450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5240ed73-6f6c-461e-b45f-99f12074fef0 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:32.376808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 710/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:32.381908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13077caa-b465-42c8-b020-b603cd5f9478 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:32.382317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13077caa-b465-42c8-b020-b603cd5f9478 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:32.382470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13077caa-b465-42c8-b020-b603cd5f9478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:32.400257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-13077caa-b465-42c8-b020-b603cd5f9478 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:32.401433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-13077caa-b465-42c8-b020-b603cd5f9478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:32.409664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 708/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:32.556741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240ed73-6f6c-461e-b45f-99f12074fef0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:32.570767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5240ed73-6f6c-461e-b45f-99f12074fef0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:32.576568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8efeec-3c7a-424f-839d-c9c62163ed72 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:32.577197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8efeec-3c7a-424f-839d-c9c62163ed72 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:32.577475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8efeec-3c7a-424f-839d-c9c62163ed72 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:32.577749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8efeec-3c7a-424f-839d-c9c62163ed72 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:32.585619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8efeec-3c7a-424f-839d-c9c62163ed72 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:32.587184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8efeec-3c7a-424f-839d-c9c62163ed72 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:32.596894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 711/1419] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:33.466939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6d3f3-2edf-4bc2-bd11-29835ad55c78 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:33.467529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6d3f3-2edf-4bc2-bd11-29835ad55c78 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:33.467529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f6d3f3-2edf-4bc2-bd11-29835ad55c78 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:33.468908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 709/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:33.473789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998691d9-bec1-4dc8-87a2-7d477f1683a5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:33.474247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998691d9-bec1-4dc8-87a2-7d477f1683a5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:33.474737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998691d9-bec1-4dc8-87a2-7d477f1683a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:33.492626 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-998691d9-bec1-4dc8-87a2-7d477f1683a5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:33.493862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-998691d9-bec1-4dc8-87a2-7d477f1683a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:33.502587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 712/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:33.651020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6d3f3-2edf-4bc2-bd11-29835ad55c78 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:33.665525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f6d3f3-2edf-4bc2-bd11-29835ad55c78 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:33.671759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f386b77f-5536-4d8d-9783-d2d11210979a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:33.672403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386b77f-5536-4d8d-9783-d2d11210979a service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:33.672685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386b77f-5536-4d8d-9783-d2d11210979a service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:33.672960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f386b77f-5536-4d8d-9783-d2d11210979a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:33.681086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386b77f-5536-4d8d-9783-d2d11210979a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:33.682975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f386b77f-5536-4d8d-9783-d2d11210979a service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:33.692207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 710/1422] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:34.546182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06988078-30b1-4a93-ac1d-eee63e0f1b9c None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:34.546615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06988078-30b1-4a93-ac1d-eee63e0f1b9c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:34.546721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06988078-30b1-4a93-ac1d-eee63e0f1b9c None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:34.548388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 713/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:34.553259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989311f-51ad-4f3f-8de4-442f32c5fd75 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:34.553662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989311f-51ad-4f3f-8de4-442f32c5fd75 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:34.553875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0989311f-51ad-4f3f-8de4-442f32c5fd75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:34.571502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0989311f-51ad-4f3f-8de4-442f32c5fd75 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:34.572728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0989311f-51ad-4f3f-8de4-442f32c5fd75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:34.580978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 711/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:34.730112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06988078-30b1-4a93-ac1d-eee63e0f1b9c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:34.744228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06988078-30b1-4a93-ac1d-eee63e0f1b9c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:34.749534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d09621-7993-464f-9ab3-8d5d25ed722f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:34.750163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d09621-7993-464f-9ab3-8d5d25ed722f service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:34.750444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d09621-7993-464f-9ab3-8d5d25ed722f service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:34.750721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d09621-7993-464f-9ab3-8d5d25ed722f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:34.758736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d09621-7993-464f-9ab3-8d5d25ed722f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:34.760343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d09621-7993-464f-9ab3-8d5d25ed722f service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:34.770155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 714/1425] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:35.618269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f20f6-70f7-4bf7-aab9-d44c22595f1d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:35.618817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f20f6-70f7-4bf7-aab9-d44c22595f1d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:35.618817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5f20f6-70f7-4bf7-aab9-d44c22595f1d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:35.620279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 712/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:35.625924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c15cc8-cc26-4e2f-a08b-b64ff87a2963 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:35.626307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c15cc8-cc26-4e2f-a08b-b64ff87a2963 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:35.626481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c15cc8-cc26-4e2f-a08b-b64ff87a2963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:35.644334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-60c15cc8-cc26-4e2f-a08b-b64ff87a2963 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:35.645533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-60c15cc8-cc26-4e2f-a08b-b64ff87a2963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:35.653248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 715/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:35.804686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f20f6-70f7-4bf7-aab9-d44c22595f1d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:35.819065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5f20f6-70f7-4bf7-aab9-d44c22595f1d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:35.824494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a988e5f4-87a1-4ddb-a9d6-c87bf4fc7033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:35.825137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988e5f4-87a1-4ddb-a9d6-c87bf4fc7033 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:35.825421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988e5f4-87a1-4ddb-a9d6-c87bf4fc7033 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:35.825701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a988e5f4-87a1-4ddb-a9d6-c87bf4fc7033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:35.834150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988e5f4-87a1-4ddb-a9d6-c87bf4fc7033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:35.835716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a988e5f4-87a1-4ddb-a9d6-c87bf4fc7033 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:35.845568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 713/1428] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:36.646545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6713647-631c-49fd-ad15-d0ad5f4c9074 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:36.647093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6713647-631c-49fd-ad15-d0ad5f4c9074 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:36.647093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6713647-631c-49fd-ad15-d0ad5f4c9074 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:36.648513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 716/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:36.653252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d630b-689a-4a35-9256-152dfc516198 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:36.653628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d630b-689a-4a35-9256-152dfc516198 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:36.653807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d630b-689a-4a35-9256-152dfc516198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:36.671365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-530d630b-689a-4a35-9256-152dfc516198 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:36.672592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-530d630b-689a-4a35-9256-152dfc516198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:36.680705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 714/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:36.829207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6713647-631c-49fd-ad15-d0ad5f4c9074 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:36.843243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6713647-631c-49fd-ad15-d0ad5f4c9074 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:36.848538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9671cfb7-9c1d-4275-89dc-0eb0f92f8710 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:36.849170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cfb7-9c1d-4275-89dc-0eb0f92f8710 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:36.849451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cfb7-9c1d-4275-89dc-0eb0f92f8710 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:36.849737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9671cfb7-9c1d-4275-89dc-0eb0f92f8710 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:36.858202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671cfb7-9c1d-4275-89dc-0eb0f92f8710 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:36.859794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9671cfb7-9c1d-4275-89dc-0eb0f92f8710 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:36.869019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 717/1431] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:37.670810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861646-05a3-4992-b5a9-3416509b48a8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:37.671387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861646-05a3-4992-b5a9-3416509b48a8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:37.671387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52861646-05a3-4992-b5a9-3416509b48a8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:37.672779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 715/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:37.677892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5247a2-9ce0-4ddc-8a1f-f626eec8189e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:37.678288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5247a2-9ce0-4ddc-8a1f-f626eec8189e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:37.678452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5247a2-9ce0-4ddc-8a1f-f626eec8189e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:37.696029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1e5247a2-9ce0-4ddc-8a1f-f626eec8189e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:37.697224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1e5247a2-9ce0-4ddc-8a1f-f626eec8189e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:37.705422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 718/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:37.855729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861646-05a3-4992-b5a9-3416509b48a8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:37.870026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52861646-05a3-4992-b5a9-3416509b48a8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:37.875485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf9685b-a77e-4398-9585-e5ddbd530fd4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:37.876159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9685b-a77e-4398-9585-e5ddbd530fd4 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:37.876453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9685b-a77e-4398-9585-e5ddbd530fd4 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:37.876735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf9685b-a77e-4398-9585-e5ddbd530fd4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:37.884899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9685b-a77e-4398-9585-e5ddbd530fd4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:37.887081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf9685b-a77e-4398-9585-e5ddbd530fd4 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:37.896428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 716/1434] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:38.691473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dc60d-da92-4e1d-bb2e-240ec4246c55 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:38.692034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dc60d-da92-4e1d-bb2e-240ec4246c55 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:38.692130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dc60d-da92-4e1d-bb2e-240ec4246c55 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:38.693425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 719/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:38.698238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb79267-ad14-4ea2-a052-8ca8220cb254 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:38.699199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb79267-ad14-4ea2-a052-8ca8220cb254 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:38.699312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb79267-ad14-4ea2-a052-8ca8220cb254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:38.717277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cfb79267-ad14-4ea2-a052-8ca8220cb254 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:38.718458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cfb79267-ad14-4ea2-a052-8ca8220cb254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:38.726111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 717/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:38.874960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dc60d-da92-4e1d-bb2e-240ec4246c55 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:38.889957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dc60d-da92-4e1d-bb2e-240ec4246c55 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:38.895401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2175206e-ce25-439c-a581-dadc6552cf44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:38.896101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175206e-ce25-439c-a581-dadc6552cf44 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:38.896382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175206e-ce25-439c-a581-dadc6552cf44 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:38.896655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2175206e-ce25-439c-a581-dadc6552cf44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:38.904801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175206e-ce25-439c-a581-dadc6552cf44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:38.906366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2175206e-ce25-439c-a581-dadc6552cf44 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:38.915655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 720/1437] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:39.712368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d704f-ae73-4cb2-b981-a91598f210aa None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:39.712920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d704f-ae73-4cb2-b981-a91598f210aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:39.712920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722d704f-ae73-4cb2-b981-a91598f210aa None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:39.714293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 718/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:19:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:39.719684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0eefa-a73a-4746-addf-459399e82c84 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:39.720159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0eefa-a73a-4746-addf-459399e82c84 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:39.720266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd0eefa-a73a-4746-addf-459399e82c84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:39.738222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4fd0eefa-a73a-4746-addf-459399e82c84 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:39.739406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4fd0eefa-a73a-4746-addf-459399e82c84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:39.747092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 721/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:39.896884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d704f-ae73-4cb2-b981-a91598f210aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:39.911459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722d704f-ae73-4cb2-b981-a91598f210aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:39.916779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80095bf-67b8-4440-b522-b97ab2100361 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:39.917414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80095bf-67b8-4440-b522-b97ab2100361 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:39.917726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80095bf-67b8-4440-b522-b97ab2100361 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:39.918005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80095bf-67b8-4440-b522-b97ab2100361 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:39.926202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80095bf-67b8-4440-b522-b97ab2100361 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:39.927786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80095bf-67b8-4440-b522-b97ab2100361 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:39.937026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 719/1440] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:41.868787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e02165-b060-4090-9751-fd64408bd98d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:41.869355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e02165-b060-4090-9751-fd64408bd98d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:41.869355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e02165-b060-4090-9751-fd64408bd98d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:41.870699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 722/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:41.876379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc30191-a3eb-4195-92bd-e4d8d30af675 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:41.876815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc30191-a3eb-4195-92bd-e4d8d30af675 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:41.877010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc30191-a3eb-4195-92bd-e4d8d30af675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:41.894715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4cc30191-a3eb-4195-92bd-e4d8d30af675 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:41.895940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4cc30191-a3eb-4195-92bd-e4d8d30af675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:41.903598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 720/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:41.941651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e02165-b060-4090-9751-fd64408bd98d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:41.943305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-54e02165-b060-4090-9751-fd64408bd98d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:41.956599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e02165-b060-4090-9751-fd64408bd98d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:41.960020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6727c7-7c21-4607-844a-06be7a901c7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:41.960645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6727c7-7c21-4607-844a-06be7a901c7c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:41.960923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6727c7-7c21-4607-844a-06be7a901c7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:41.961187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6727c7-7c21-4607-844a-06be7a901c7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:41.970472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6727c7-7c21-4607-844a-06be7a901c7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:41.971971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6727c7-7c21-4607-844a-06be7a901c7c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:41.981393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 723/1443] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:19:44.342035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494f9e-f77b-4be4-ade9-3812a816f8fe None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:44.342604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494f9e-f77b-4be4-ade9-3812a816f8fe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:44.342604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc494f9e-f77b-4be4-ade9-3812a816f8fe None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:44.343989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 721/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:44.349181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefc4ea-362a-4adc-999c-158c767b105d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:44.349574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefc4ea-362a-4adc-999c-158c767b105d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:44.349734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edefc4ea-362a-4adc-999c-158c767b105d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:44.367416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-edefc4ea-362a-4adc-999c-158c767b105d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:44.368638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-edefc4ea-362a-4adc-999c-158c767b105d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:44.376985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 724/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:44.413833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc494f9e-f77b-4be4-ade9-3812a816f8fe None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:44.425332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc494f9e-f77b-4be4-ade9-3812a816f8fe None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:44.428826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a929d54f-f388-40f2-be77-8ecefb27a9e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:44.429460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929d54f-f388-40f2-be77-8ecefb27a9e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:44.429746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929d54f-f388-40f2-be77-8ecefb27a9e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:44.430024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a929d54f-f388-40f2-be77-8ecefb27a9e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:44.439508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929d54f-f388-40f2-be77-8ecefb27a9e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:44.440980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a929d54f-f388-40f2-be77-8ecefb27a9e5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:44.450203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 722/1446] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:46.758415 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d680d2-209e-4a23-abe8-0e812a6416f4 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:46.758978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d680d2-209e-4a23-abe8-0e812a6416f4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:46.758978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d680d2-209e-4a23-abe8-0e812a6416f4 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:46.760362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 725/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:46.765653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8215bc0-15e1-4fad-a5ae-468e3e89ccb1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:46.766063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8215bc0-15e1-4fad-a5ae-468e3e89ccb1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:46.766214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8215bc0-15e1-4fad-a5ae-468e3e89ccb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:46.783584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b8215bc0-15e1-4fad-a5ae-468e3e89ccb1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:46.784768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b8215bc0-15e1-4fad-a5ae-468e3e89ccb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:46.792541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 723/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:46.831172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d680d2-209e-4a23-abe8-0e812a6416f4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:46.841706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d680d2-209e-4a23-abe8-0e812a6416f4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:46.845205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29891521-94be-452b-841a-d2169275acee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:46.845832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29891521-94be-452b-841a-d2169275acee service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:46.846113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29891521-94be-452b-841a-d2169275acee service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:46.846386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29891521-94be-452b-841a-d2169275acee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:46.855055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29891521-94be-452b-841a-d2169275acee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:46.856999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29891521-94be-452b-841a-d2169275acee service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:46.866170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 726/1449] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:19:49.009685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926acb2d-a79b-449c-8fb7-7981c21a3743 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:49.010252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926acb2d-a79b-449c-8fb7-7981c21a3743 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:49.010252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926acb2d-a79b-449c-8fb7-7981c21a3743 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:49.011653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 724/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:49.017347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f97e93-d3a3-419f-a6e6-36dcfda04af1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:49.017759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f97e93-d3a3-419f-a6e6-36dcfda04af1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:49.017941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f97e93-d3a3-419f-a6e6-36dcfda04af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:49.036257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e6f97e93-d3a3-419f-a6e6-36dcfda04af1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:49.037465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6f97e93-d3a3-419f-a6e6-36dcfda04af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:49.045518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 727/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:49.083731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926acb2d-a79b-449c-8fb7-7981c21a3743 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:49.095059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926acb2d-a79b-449c-8fb7-7981c21a3743 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:49.098528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2709fd-a0cb-4820-aab7-a098c229e5ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:49.099163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2709fd-a0cb-4820-aab7-a098c229e5ba service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:49.099445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2709fd-a0cb-4820-aab7-a098c229e5ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:49.099720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2709fd-a0cb-4820-aab7-a098c229e5ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:49.108368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2709fd-a0cb-4820-aab7-a098c229e5ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:49.109788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2709fd-a0cb-4820-aab7-a098c229e5ba service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:49.118939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 725/1452] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:51.197525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600d3cf-46c8-47d1-9ead-3a7c7769887f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:51.198051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600d3cf-46c8-47d1-9ead-3a7c7769887f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:51.198051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6600d3cf-46c8-47d1-9ead-3a7c7769887f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:51.199379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 728/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:51.205917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3be7f-f320-4c6d-9887-775be35dc9d8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:51.206313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3be7f-f320-4c6d-9887-775be35dc9d8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:51.206468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af3be7f-f320-4c6d-9887-775be35dc9d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:51.226629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7af3be7f-f320-4c6d-9887-775be35dc9d8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:51.227847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7af3be7f-f320-4c6d-9887-775be35dc9d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:51.235985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 726/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:51.274105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600d3cf-46c8-47d1-9ead-3a7c7769887f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:51.285045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6600d3cf-46c8-47d1-9ead-3a7c7769887f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:51.288587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5014d6ab-34cb-43f3-a830-37aecee14300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:51.289227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014d6ab-34cb-43f3-a830-37aecee14300 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:51.289511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014d6ab-34cb-43f3-a830-37aecee14300 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:51.289791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5014d6ab-34cb-43f3-a830-37aecee14300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:51.300198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014d6ab-34cb-43f3-a830-37aecee14300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:51.301624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5014d6ab-34cb-43f3-a830-37aecee14300 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:51.311518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 729/1455] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:53.367679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcc5a7-2014-4a29-a453-81e5054b9cb6 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:53.368263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcc5a7-2014-4a29-a453-81e5054b9cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:53.368263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fcc5a7-2014-4a29-a453-81e5054b9cb6 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:53.369642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 727/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:53.375281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33e05-e2d8-41c6-b643-b448494b877c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:53.375721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33e05-e2d8-41c6-b643-b448494b877c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:53.375863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e33e05-e2d8-41c6-b643-b448494b877c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:53.393631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c7e33e05-e2d8-41c6-b643-b448494b877c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:53.394832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c7e33e05-e2d8-41c6-b643-b448494b877c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:53.403035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 730/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:53.440907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcc5a7-2014-4a29-a453-81e5054b9cb6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:53.451546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fcc5a7-2014-4a29-a453-81e5054b9cb6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:53.455247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f743c41b-271f-4aa9-946f-8021b7d11c9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:53.455918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743c41b-271f-4aa9-946f-8021b7d11c9a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:53.456206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743c41b-271f-4aa9-946f-8021b7d11c9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:53.456486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f743c41b-271f-4aa9-946f-8021b7d11c9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:53.465228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743c41b-271f-4aa9-946f-8021b7d11c9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:53.466682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f743c41b-271f-4aa9-946f-8021b7d11c9a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:53.476031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 728/1458] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:55.541661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a2f1-00d7-4621-8851-b7c4f9892287 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:55.542138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a2f1-00d7-4621-8851-b7c4f9892287 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:55.542261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a2f1-00d7-4621-8851-b7c4f9892287 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:55.543624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 731/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:55.549704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6dd88-ad5a-44ac-a3b7-c4db164436d3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:55.550357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6dd88-ad5a-44ac-a3b7-c4db164436d3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:55.550514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e6dd88-ad5a-44ac-a3b7-c4db164436d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:55.567816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b7e6dd88-ad5a-44ac-a3b7-c4db164436d3 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:55.569014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b7e6dd88-ad5a-44ac-a3b7-c4db164436d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:55.577317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 729/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:55.614152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9a2f1-00d7-4621-8851-b7c4f9892287 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:55.624779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9a2f1-00d7-4621-8851-b7c4f9892287 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:55.628231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa986b8c-2f2f-45e7-b2da-b540108951d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:55.628867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa986b8c-2f2f-45e7-b2da-b540108951d2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:55.629150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa986b8c-2f2f-45e7-b2da-b540108951d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:55.629423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa986b8c-2f2f-45e7-b2da-b540108951d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:55.638667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa986b8c-2f2f-45e7-b2da-b540108951d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:55.640135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa986b8c-2f2f-45e7-b2da-b540108951d2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:55.649262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 732/1461] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:57.689494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7fb3-bf67-4cee-8c65-5bca35badd78 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:57.690025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7fb3-bf67-4cee-8c65-5bca35badd78 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:57.690025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daf7fb3-bf67-4cee-8c65-5bca35badd78 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:57.691413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 730/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:57.696878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801607-3e67-4811-862d-7f3518467cd1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:57.697284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801607-3e67-4811-862d-7f3518467cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:57.697438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5801607-3e67-4811-862d-7f3518467cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:57.703368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=79 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:19:57.715611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f5801607-3e67-4811-862d-7f3518467cd1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:57.717384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f5801607-3e67-4811-862d-7f3518467cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:57.725181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 733/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:57.762465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7fb3-bf67-4cee-8c65-5bca35badd78 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:57.767951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=73 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:19:57.774355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daf7fb3-bf67-4cee-8c65-5bca35badd78 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:57.777887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64809784-5cd9-4640-8382-486d98b64da0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:57.779060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809784-5cd9-4640-8382-486d98b64da0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:57.779345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809784-5cd9-4640-8382-486d98b64da0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:57.779624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64809784-5cd9-4640-8382-486d98b64da0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:57.788496 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64809784-5cd9-4640-8382-486d98b64da0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:57.789927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64809784-5cd9-4640-8382-486d98b64da0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:57.799736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 731/1464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:19:59.831680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75477c-58fe-4e0a-9b0b-c7dd04e4bff6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:59.832254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75477c-58fe-4e0a-9b0b-c7dd04e4bff6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:59.832254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a75477c-58fe-4e0a-9b0b-c7dd04e4bff6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:59.833630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 734/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:19:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:19:59.838904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa662da8-64ee-48e8-9a48-6c20ea877479 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:59.839314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa662da8-64ee-48e8-9a48-6c20ea877479 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:59.839489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa662da8-64ee-48e8-9a48-6c20ea877479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:59.857468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fa662da8-64ee-48e8-9a48-6c20ea877479 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:19:59.858664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fa662da8-64ee-48e8-9a48-6c20ea877479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:19:59.866442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 732/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:19:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:19:59.902577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75477c-58fe-4e0a-9b0b-c7dd04e4bff6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:19:59.913149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a75477c-58fe-4e0a-9b0b-c7dd04e4bff6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:19:59.916627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca4a03d-6147-4620-9734-92a993e15438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:19:59.917256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4a03d-6147-4620-9734-92a993e15438 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:19:59.917538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4a03d-6147-4620-9734-92a993e15438 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:19:59.917817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca4a03d-6147-4620-9734-92a993e15438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:19:59.926768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4a03d-6147-4620-9734-92a993e15438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:19:59.928236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca4a03d-6147-4620-9734-92a993e15438 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:19:59.937994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 735/1467] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:19:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:01.979802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf18852-7e1f-43e8-9307-fd05f8b4dc62 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:01.980369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf18852-7e1f-43e8-9307-fd05f8b4dc62 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:01.980369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf18852-7e1f-43e8-9307-fd05f8b4dc62 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:01.982522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 733/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:20:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:01.988016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54defed-a8d3-4025-8450-e2dbd8d364b1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:01.988441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54defed-a8d3-4025-8450-e2dbd8d364b1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:01.988574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54defed-a8d3-4025-8450-e2dbd8d364b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:02.005954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e54defed-a8d3-4025-8450-e2dbd8d364b1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:02.007148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e54defed-a8d3-4025-8450-e2dbd8d364b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:02.015182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 736/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:20:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:02.052268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf18852-7e1f-43e8-9307-fd05f8b4dc62 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:02.062755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf18852-7e1f-43e8-9307-fd05f8b4dc62 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:02.066260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7208b8c3-5e16-493f-a598-147c0cf190b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:02.066900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208b8c3-5e16-493f-a598-147c0cf190b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:02.067186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208b8c3-5e16-493f-a598-147c0cf190b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:02.067466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7208b8c3-5e16-493f-a598-147c0cf190b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:02.076234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208b8c3-5e16-493f-a598-147c0cf190b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:02.077661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7208b8c3-5e16-493f-a598-147c0cf190b5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:02.086911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 734/1470] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:20:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:12.000899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=11 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:20:12.082795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=14 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:20:21.625538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b1068-daef-44f9-be8d-9769d8ccfb6e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:21.626096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b1068-daef-44f9-be8d-9769d8ccfb6e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:21.626096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928b1068-daef-44f9-be8d-9769d8ccfb6e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:21.627447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 737/1471] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:20:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:21.632826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1b850-97ce-453f-a6ed-7c60f0917eae None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:21.633304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1b850-97ce-453f-a6ed-7c60f0917eae None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:21.633411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e1b850-97ce-453f-a6ed-7c60f0917eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:21.651265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-47e1b850-97ce-453f-a6ed-7c60f0917eae None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:21.652490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-47e1b850-97ce-453f-a6ed-7c60f0917eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:21.660177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 735/1472] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:20:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:21.741355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e136e5-2678-4ee4-a50f-f27d77048cd3 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:21.741651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e136e5-2678-4ee4-a50f-f27d77048cd3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:21.741937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e136e5-2678-4ee4-a50f-f27d77048cd3 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:21.743292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 738/1473] 10.222.0.16 () {54 vars in 962 bytes} [Thu Jul 7 10:20:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:21.748119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235cf16-b931-49a2-86b4-bbacc1c8464a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:21.748413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235cf16-b931-49a2-86b4-bbacc1c8464a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:21.748690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235cf16-b931-49a2-86b4-bbacc1c8464a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:21.767343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6235cf16-b931-49a2-86b4-bbacc1c8464a None None] MFA Rules not processed for user `4b320f5e6fc4489484fe6599f4391245`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:21.768610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6235cf16-b931-49a2-86b4-bbacc1c8464a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:21.774899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 736/1474] 10.222.0.16 () {58 vars in 1059 bytes} [Thu Jul 7 10:20:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:21.782063 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e136e5-2678-4ee4-a50f-f27d77048cd3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:21.792711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e136e5-2678-4ee4-a50f-f27d77048cd3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:21.796478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19faa76f-e4d0-4afe-8ad3-0b8b253ed7ed service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:21.797105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faa76f-e4d0-4afe-8ad3-0b8b253ed7ed service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:21.797388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faa76f-e4d0-4afe-8ad3-0b8b253ed7ed service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:21.797663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19faa76f-e4d0-4afe-8ad3-0b8b253ed7ed service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:21.807026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19faa76f-e4d0-4afe-8ad3-0b8b253ed7ed service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:21.808480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19faa76f-e4d0-4afe-8ad3-0b8b253ed7ed service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:21.818326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 739/1475] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:20:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:23.919412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892c6ce-5b05-4185-a81b-836d31ee6a07 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:23.920020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892c6ce-5b05-4185-a81b-836d31ee6a07 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:23.920020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0892c6ce-5b05-4185-a81b-836d31ee6a07 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:23.921415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 737/1476] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:20:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:23.926658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628a852-968d-468e-b8da-9009763fa78d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:23.927088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628a852-968d-468e-b8da-9009763fa78d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:23.927247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7628a852-968d-468e-b8da-9009763fa78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:23.945954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7628a852-968d-468e-b8da-9009763fa78d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:23.947137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7628a852-968d-468e-b8da-9009763fa78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:23.955533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 740/1477] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:20:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:23.978075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892c6ce-5b05-4185-a81b-836d31ee6a07 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:23.990248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0892c6ce-5b05-4185-a81b-836d31ee6a07 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:23.994600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7c5769-50b3-4a52-a644-23ff61b4ae44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:23.995239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c5769-50b3-4a52-a644-23ff61b4ae44 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:23.995524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c5769-50b3-4a52-a644-23ff61b4ae44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:23.995841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7c5769-50b3-4a52-a644-23ff61b4ae44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:23.996566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c5769-50b3-4a52-a644-23ff61b4ae44 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:23.998344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-bf7c5769-50b3-4a52-a644-23ff61b4ae44 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:20:23.998344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:20:23.998344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:20:24.000025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7c5769-50b3-4a52-a644-23ff61b4ae44 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:24.019156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 738/1478] 10.222.0.16 () {56 vars in 1108 bytes} [Thu Jul 7 10:20:23 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:20:26.048142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd8cf-dd82-47d2-b6cc-a4660db8c20f None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:26.048686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd8cf-dd82-47d2-b6cc-a4660db8c20f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:26.048686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0bd8cf-dd82-47d2-b6cc-a4660db8c20f None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:26.050057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 741/1479] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:26.055474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36281147-b47f-46cc-a97c-c7c57d2d43f7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:26.055935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36281147-b47f-46cc-a97c-c7c57d2d43f7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:26.056069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36281147-b47f-46cc-a97c-c7c57d2d43f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:26.073963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-36281147-b47f-46cc-a97c-c7c57d2d43f7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:26.075155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36281147-b47f-46cc-a97c-c7c57d2d43f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:26.083383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 739/1480] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:20:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:26.125069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd8cf-dd82-47d2-b6cc-a4660db8c20f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:26.139322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0bd8cf-dd82-47d2-b6cc-a4660db8c20f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:26.145201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0842a45d-32f7-437d-8957-09256b5ff5bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:26.145829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842a45d-32f7-437d-8957-09256b5ff5bc service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:26.146110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842a45d-32f7-437d-8957-09256b5ff5bc service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:26.146384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0842a45d-32f7-437d-8957-09256b5ff5bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:26.154279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842a45d-32f7-437d-8957-09256b5ff5bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:26.155914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0842a45d-32f7-437d-8957-09256b5ff5bc service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:26.165081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 742/1481] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:28.346774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943667de-49c8-4324-b58c-15e09d4e5837 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:28.347346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943667de-49c8-4324-b58c-15e09d4e5837 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:28.347346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943667de-49c8-4324-b58c-15e09d4e5837 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:28.348747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 740/1482] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:20:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:28.353787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e6513-7853-45bf-aa13-8e1ac4250bde None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:28.354163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e6513-7853-45bf-aa13-8e1ac4250bde None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:28.354332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e6513-7853-45bf-aa13-8e1ac4250bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:28.371966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-013e6513-7853-45bf-aa13-8e1ac4250bde None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:28.373149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-013e6513-7853-45bf-aa13-8e1ac4250bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:28.380765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 743/1483] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:20:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:28.415700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943667de-49c8-4324-b58c-15e09d4e5837 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:28.426859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943667de-49c8-4324-b58c-15e09d4e5837 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:28.430602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27a568e-501a-44cf-968c-5e3b6872aa72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:28.431236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a568e-501a-44cf-968c-5e3b6872aa72 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:28.431524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a568e-501a-44cf-968c-5e3b6872aa72 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:28.431835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27a568e-501a-44cf-968c-5e3b6872aa72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:28.440727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a568e-501a-44cf-968c-5e3b6872aa72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:28.442152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27a568e-501a-44cf-968c-5e3b6872aa72 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:28.451487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 741/1484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:20:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:30.530080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c139-6d6b-41c4-bf69-abca3f28f235 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:30.530532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c139-6d6b-41c4-bf69-abca3f28f235 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:30.530638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1560c139-6d6b-41c4-bf69-abca3f28f235 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:30.532017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 744/1485] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:30.537096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ccc1-5eea-4821-b348-2a37140f37f2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:30.537531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ccc1-5eea-4821-b348-2a37140f37f2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:30.537663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5970ccc1-5eea-4821-b348-2a37140f37f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:30.555362 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5970ccc1-5eea-4821-b348-2a37140f37f2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:30.556605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5970ccc1-5eea-4821-b348-2a37140f37f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:30.564898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 742/1486] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:20:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:30.602489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c139-6d6b-41c4-bf69-abca3f28f235 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:30.613133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1560c139-6d6b-41c4-bf69-abca3f28f235 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:30.616591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd74417b-fd61-47f4-bbab-8fc50cb0c2c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:30.617220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74417b-fd61-47f4-bbab-8fc50cb0c2c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:30.617504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74417b-fd61-47f4-bbab-8fc50cb0c2c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:30.617782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd74417b-fd61-47f4-bbab-8fc50cb0c2c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:30.626986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74417b-fd61-47f4-bbab-8fc50cb0c2c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:30.628447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd74417b-fd61-47f4-bbab-8fc50cb0c2c2 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:30.637678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 745/1487] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:20:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:32.798033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ccce5-c11e-40a0-9c80-efaf16eb889d None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:32.798567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ccce5-c11e-40a0-9c80-efaf16eb889d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:32.798567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8ccce5-c11e-40a0-9c80-efaf16eb889d None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:32.799970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 743/1488] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:20:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:32.805389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df6ee8-2cde-4dd8-a9ac-a8b6385d9ec8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:32.805791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df6ee8-2cde-4dd8-a9ac-a8b6385d9ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:32.805943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26df6ee8-2cde-4dd8-a9ac-a8b6385d9ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:32.822986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-26df6ee8-2cde-4dd8-a9ac-a8b6385d9ec8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:32.824197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-26df6ee8-2cde-4dd8-a9ac-a8b6385d9ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:32.831719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 746/1489] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:32.871214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ccce5-c11e-40a0-9c80-efaf16eb889d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:32.881732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8ccce5-c11e-40a0-9c80-efaf16eb889d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:32.885252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc3109c-ed2e-4193-af2d-8d3049ee14fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:32.885891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3109c-ed2e-4193-af2d-8d3049ee14fd service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:32.886178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3109c-ed2e-4193-af2d-8d3049ee14fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:32.886459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc3109c-ed2e-4193-af2d-8d3049ee14fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:32.895184 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3109c-ed2e-4193-af2d-8d3049ee14fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:32.896756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc3109c-ed2e-4193-af2d-8d3049ee14fd service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:32.906930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 744/1490] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:35.085736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e4f2d-b48e-42b8-92d0-5febdc75df41 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:35.086264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e4f2d-b48e-42b8-92d0-5febdc75df41 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:35.086383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e4f2d-b48e-42b8-92d0-5febdc75df41 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:35.087827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 747/1491] 10.222.0.16 () {54 vars in 926 bytes} [Thu Jul 7 10:20:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:20:35.093108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f0cee-1fdd-4bbc-9fd0-439ae6a33b72 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:35.093568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f0cee-1fdd-4bbc-9fd0-439ae6a33b72 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:35.093699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f0cee-1fdd-4bbc-9fd0-439ae6a33b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:35.110867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-073f0cee-1fdd-4bbc-9fd0-439ae6a33b72 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:20:35.112100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-073f0cee-1fdd-4bbc-9fd0-439ae6a33b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:20:35.119667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 745/1492] 10.222.0.16 () {58 vars in 1023 bytes} [Thu Jul 7 10:20:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:20:35.158661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e4f2d-b48e-42b8-92d0-5febdc75df41 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:20:35.169636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e4f2d-b48e-42b8-92d0-5febdc75df41 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:20:35.173184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1ee99a-c90d-48ae-adac-e9d73919985f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:20:35.173828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ee99a-c90d-48ae-adac-e9d73919985f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:20:35.174111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ee99a-c90d-48ae-adac-e9d73919985f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:20:35.174389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1ee99a-c90d-48ae-adac-e9d73919985f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:20:35.183558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ee99a-c90d-48ae-adac-e9d73919985f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:20:35.185027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1ee99a-c90d-48ae-adac-e9d73919985f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:20:35.195091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 748/1493] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:20:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:20:45.105820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=33 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:20:45.190791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=33 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:22:00.564247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7281d1-aa4e-4b68-9743-0f1abe0717f6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.564784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7281d1-aa4e-4b68-9743-0f1abe0717f6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.564885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7281d1-aa4e-4b68-9743-0f1abe0717f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.583581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4f7281d1-aa4e-4b68-9743-0f1abe0717f6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:22:00.584905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f7281d1-aa4e-4b68-9743-0f1abe0717f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:22:00.592667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 746/1494] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:22:00.600456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537a8c7-140e-4513-83ef-1ad0b01b100f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.600918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537a8c7-140e-4513-83ef-1ad0b01b100f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.601022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537a8c7-140e-4513-83ef-1ad0b01b100f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.618281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2537a8c7-140e-4513-83ef-1ad0b01b100f None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:22:00.619462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2537a8c7-140e-4513-83ef-1ad0b01b100f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:22:00.627148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 749/1495] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:22:00.631939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7281d1-aa4e-4b68-9743-0f1abe0717f6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:00.641373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7281d1-aa4e-4b68-9743-0f1abe0717f6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:00.646340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:00.646992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.647279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.647558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.648309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:22:00.648618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:00.649620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:00.649620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:22:00.649620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:22:00.650508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:00.661633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 747/1496] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:22:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:22:00.669299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537a8c7-140e-4513-83ef-1ad0b01b100f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:00.678335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537a8c7-140e-4513-83ef-1ad0b01b100f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:00.682806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:00.683449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.683731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.684030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.684707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:00.686756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:00.686756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 07 10:22:00.686756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: Jul 07 10:22:00.687996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:00.724495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 750/1497] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:22:00 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:22:00.728486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:00.738320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b595148-ad36-4165-874f-0ee5230ea49f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:00.792974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:00.793573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.793862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.794138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.794829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:00.795855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:00.795855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:22:00.795855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:22:00.797061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:00.816860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 748/1498] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:22:00 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:22:00.820923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:00.830966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db650ce-ed4b-4262-8eab-4185036add1e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:00.836037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:00.836691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.836974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.837246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.837962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:00.838966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:00.838966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 07 10:22:00.838966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: Jul 07 10:22:00.839876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:00.852322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 751/1499] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:22:00.856077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:00.866265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c9825-a2ef-42e5-a5ca-9e7e22f206d1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:00.870862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73247d6-6976-4be7-961f-6b75e69627f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:00.871513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73247d6-6976-4be7-961f-6b75e69627f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.871834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73247d6-6976-4be7-961f-6b75e69627f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.872115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73247d6-6976-4be7-961f-6b75e69627f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.872837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73247d6-6976-4be7-961f-6b75e69627f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:00.873845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-b73247d6-6976-4be7-961f-6b75e69627f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:00.873845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:22:00.873845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:22:00.874734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73247d6-6976-4be7-961f-6b75e69627f6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:00.888350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 749/1500] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:22:00 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:22:00.892135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:00.902305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17bc9b-d66b-4802-9870-d1d29b8033a2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:00.906842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71e761e-11db-4d41-b9f7-fa6f53226f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:00.907707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e761e-11db-4d41-b9f7-fa6f53226f2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.908026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e761e-11db-4d41-b9f7-fa6f53226f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.908303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71e761e-11db-4d41-b9f7-fa6f53226f2a admin admin] PATH_INFO: `/v3/projects/a2588a1151a04ca2b4dfdab2bc12814d/users/8e9d2f6c2dd1472cbe8c616f12d3bcdf/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.923350 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e761e-11db-4d41-b9f7-fa6f53226f2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2588a1151a04ca2b4dfdab2bc12814d, user_id=8e9d2f6c2dd1472cbe8c616f12d3bcdf, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:00.924411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-c71e761e-11db-4d41-b9f7-fa6f53226f2a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:00.924411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 07 10:22:00.924411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: Jul 07 10:22:00.926400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71e761e-11db-4d41-b9f7-fa6f53226f2a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:00.940457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 752/1501] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:22:00 2022] PUT /identity/v3/projects/a2588a1151a04ca2b4dfdab2bc12814d/users/8e9d2f6c2dd1472cbe8c616f12d3bcdf/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:22:00.952381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fd3ae-593c-48fc-ad63-e1ce527ccfeb None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:00.952681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fd3ae-593c-48fc-ad63-e1ce527ccfeb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:00.952963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80fd3ae-593c-48fc-ad63-e1ce527ccfeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:00.979317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e80fd3ae-593c-48fc-ad63-e1ce527ccfeb None None] MFA Rules not processed for user `8e9d2f6c2dd1472cbe8c616f12d3bcdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:22:00.980563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e80fd3ae-593c-48fc-ad63-e1ce527ccfeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:22:01.042175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 750/1502] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:22:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:22:01.055132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45192d2-c57a-42d5-b25c-8951500aa7e2 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.055425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45192d2-c57a-42d5-b25c-8951500aa7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.055700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45192d2-c57a-42d5-b25c-8951500aa7e2 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.057084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 753/1503] 10.222.0.16 () {54 vars in 963 bytes} [Thu Jul 7 10:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:22:01.062098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33251197-b92f-425f-b882-16505fa1af8d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.062393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33251197-b92f-425f-b882-16505fa1af8d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.062675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33251197-b92f-425f-b882-16505fa1af8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.081123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-33251197-b92f-425f-b882-16505fa1af8d None None] MFA Rules not processed for user `cc73785ed9da4ad297b924632b14e92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:22:01.082824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-33251197-b92f-425f-b882-16505fa1af8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:22:01.124863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:22:01.133414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 751/1504] 10.222.0.16 () {58 vars in 1060 bytes} [Thu Jul 7 10:22:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:22:01.140959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45192d2-c57a-42d5-b25c-8951500aa7e2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:01.151881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45192d2-c57a-42d5-b25c-8951500aa7e2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:01.155290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:01.155947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.156232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.156504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.165363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:01.166779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:01.176189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 754/1505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:22:01 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:22:01.217858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340a9f-6e6a-4f29-a8d3-32e1ad913da8 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.218150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340a9f-6e6a-4f29-a8d3-32e1ad913da8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.218458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87340a9f-6e6a-4f29-a8d3-32e1ad913da8 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.219753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 752/1506] 10.222.0.16 () {52 vars in 829 bytes} [Thu Jul 7 10:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:22:01.223780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:01.287329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32466b1e-2928-4c9b-8def-a6ca6503b511 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:01.291841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:01.292714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.292997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.293275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] PATH_INFO: `/v3/users/8e9d2f6c2dd1472cbe8c616f12d3bcdf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.294449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e9d2f6c2dd1472cbe8c616f12d3bcdf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:01.295451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:01.295451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 07 10:22:01.295451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: Jul 07 10:22:01.296692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:01.360067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e9d2f6c2dd1472cbe8c616f12d3bcdf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:01.369205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e9d2f6c2dd1472cbe8c616f12d3bcdf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:01.369837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e9d2f6c2dd1472cbe8c616f12d3bcdf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:01.382350 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e9d2f6c2dd1472cbe8c616f12d3bcdf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:01.398594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:22:01.401259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e9d2f6c2dd1472cbe8c616f12d3bcdf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:01.408484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 755/1507] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:22:01 2022] DELETE /identity/v3/users/8e9d2f6c2dd1472cbe8c616f12d3bcdf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:22:01.412976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.413296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.413598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.433442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:22:01.434766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:22:01.494444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 753/1508] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:22:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:22:01.504492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:01.505935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:22:01.564269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e678e57-a164-4513-9360-78823afee980 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:01.568264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad9b7b-7c4e-4dee-93a4-a60d7ba0a618 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:01.568926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9b7b-7c4e-4dee-93a4-a60d7ba0a618 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.569210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9b7b-7c4e-4dee-93a4-a60d7ba0a618 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.569486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad9b7b-7c4e-4dee-93a4-a60d7ba0a618 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.578248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad9b7b-7c4e-4dee-93a4-a60d7ba0a618 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:01.579677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad9b7b-7c4e-4dee-93a4-a60d7ba0a618 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:01.589542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 756/1509] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:22:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:22:01.946226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:22:01.947776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:22:01.959617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c5cac4-8651-4abb-856c-8827d1bbffb6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:22:01.964280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:22:01.965110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:22:01.965398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:22:01.965662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] PATH_INFO: `/v3/projects/a2588a1151a04ca2b4dfdab2bc12814d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:22:01.966840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2588a1151a04ca2b4dfdab2bc12814d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:22:01.967887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:22:01.967887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:22:01.967887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:22:01.971703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:22:02.036089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2588a1151a04ca2b4dfdab2bc12814d', 'request_id': 'req-39ef5cc7-062f-44a2-b46c-1a3e7089163e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:02.046369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2588a1151a04ca2b4dfdab2bc12814d', 'request_id': 'req-39ef5cc7-062f-44a2-b46c-1a3e7089163e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:02.061764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2588a1151a04ca2b4dfdab2bc12814d', 'request_id': 'req-39ef5cc7-062f-44a2-b46c-1a3e7089163e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:02.072733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] The token cache is being invalidate because project a2588a1151a04ca2b4dfdab2bc12814d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:22:02.073027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:02.073721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-39ef5cc7-062f-44a2-b46c-1a3e7089163e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:22:02.075459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 754/1510] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:22:01 2022] DELETE /identity/v3/projects/a2588a1151a04ca2b4dfdab2bc12814d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:22:11.585437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:22:12.069762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:23:45.187884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe00e6-b0b9-460f-a1b8-e23a38d0ae58 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:45.188503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe00e6-b0b9-460f-a1b8-e23a38d0ae58 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:45.188503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe00e6-b0b9-460f-a1b8-e23a38d0ae58 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:45.189859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 757/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:23:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:23:45.195090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d2a8d-a9e2-4123-9ee0-1d3575613c27 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:45.195519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d2a8d-a9e2-4123-9ee0-1d3575613c27 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:45.195682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d2a8d-a9e2-4123-9ee0-1d3575613c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:45.214313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fb2d2a8d-a9e2-4123-9ee0-1d3575613c27 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:23:45.215539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb2d2a8d-a9e2-4123-9ee0-1d3575613c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:23:45.272241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 755/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:23:45.421757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe00e6-b0b9-460f-a1b8-e23a38d0ae58 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:23:45.423459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bcbe00e6-b0b9-460f-a1b8-e23a38d0ae58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:23:45.497098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe00e6-b0b9-460f-a1b8-e23a38d0ae58 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:23:45.502425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bdd3cb-407f-4020-a472-436856c1090e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:23:45.503067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdd3cb-407f-4020-a472-436856c1090e service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:45.503352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdd3cb-407f-4020-a472-436856c1090e service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:45.503628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bdd3cb-407f-4020-a472-436856c1090e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:45.511953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdd3cb-407f-4020-a472-436856c1090e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:23:45.513563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bdd3cb-407f-4020-a472-436856c1090e service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:23:45.523028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 758/1513] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:23:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:23:46.344937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecfa4a-dfd5-45c5-b51f-96dd44a7d986 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:46.345489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecfa4a-dfd5-45c5-b51f-96dd44a7d986 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:46.345489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffecfa4a-dfd5-45c5-b51f-96dd44a7d986 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:46.346936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 756/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:23:46.351938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c570f-46df-478a-bbe3-67cc4a50511b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:46.352366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c570f-46df-478a-bbe3-67cc4a50511b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:46.352524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5c570f-46df-478a-bbe3-67cc4a50511b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:46.370157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5f5c570f-46df-478a-bbe3-67cc4a50511b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:23:46.371347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5f5c570f-46df-478a-bbe3-67cc4a50511b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:23:46.378986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 759/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:23:46.530345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecfa4a-dfd5-45c5-b51f-96dd44a7d986 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:23:46.544727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffecfa4a-dfd5-45c5-b51f-96dd44a7d986 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:23:46.550194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570139c-7fda-4977-ac92-013d0a41b1db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:23:46.550841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570139c-7fda-4977-ac92-013d0a41b1db service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:46.551145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570139c-7fda-4977-ac92-013d0a41b1db service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:46.551424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570139c-7fda-4977-ac92-013d0a41b1db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:46.561992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570139c-7fda-4977-ac92-013d0a41b1db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:23:46.563657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570139c-7fda-4977-ac92-013d0a41b1db service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:23:46.574209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 757/1516] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:23:47.199071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146158-08c4-4031-9bb6-22a908a2a299 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:47.199387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146158-08c4-4031-9bb6-22a908a2a299 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:47.199665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9146158-08c4-4031-9bb6-22a908a2a299 None None] PATH_INFO: `/v3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:47.200999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 760/1517] 10.222.0.58 () {54 vars in 925 bytes} [Thu Jul 7 10:23:47 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:23:47.214771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b827dc-53fa-4c8a-8f35-55825b883a6b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:47.215060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b827dc-53fa-4c8a-8f35-55825b883a6b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:47.215339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b827dc-53fa-4c8a-8f35-55825b883a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:47.233489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-45b827dc-53fa-4c8a-8f35-55825b883a6b None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:23:47.234692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45b827dc-53fa-4c8a-8f35-55825b883a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:23:47.294950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 758/1518] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 7 10:23:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:23:51.123471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d617-f5cc-4f42-a366-964823582139 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:51.124062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d617-f5cc-4f42-a366-964823582139 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:51.124062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0692d617-f5cc-4f42-a366-964823582139 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:51.125358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 761/1519] 10.222.0.16 () {54 vars in 960 bytes} [Thu Jul 7 10:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:23:51.130470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931599e1-cf1e-43d4-a11a-0e47864334fe None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:51.130910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931599e1-cf1e-43d4-a11a-0e47864334fe None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:51.131071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931599e1-cf1e-43d4-a11a-0e47864334fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:51.149551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-931599e1-cf1e-43d4-a11a-0e47864334fe None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:23:51.150758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-931599e1-cf1e-43d4-a11a-0e47864334fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:23:51.159301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 759/1520] 10.222.0.16 () {58 vars in 1057 bytes} [Thu Jul 7 10:23:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:23:51.167054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d617-f5cc-4f42-a366-964823582139 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:23:51.180211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0692d617-f5cc-4f42-a366-964823582139 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:23:51.185640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1068a8-a394-4550-9147-c5ad6e12f894 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:23:51.186361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1068a8-a394-4550-9147-c5ad6e12f894 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:51.186642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1068a8-a394-4550-9147-c5ad6e12f894 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:51.186914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1068a8-a394-4550-9147-c5ad6e12f894 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:51.198528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1068a8-a394-4550-9147-c5ad6e12f894 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:23:51.200602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1068a8-a394-4550-9147-c5ad6e12f894 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:23:51.210440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 762/1521] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:23:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:23:57.797733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b11eb3-a4b1-4ca0-9730-beade63cc02c None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:57.798284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b11eb3-a4b1-4ca0-9730-beade63cc02c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:57.798284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b11eb3-a4b1-4ca0-9730-beade63cc02c None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:57.799623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 760/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:23:57.804351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6fe9-85b1-4c97-b410-eb15a38a1299 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:57.804771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6fe9-85b1-4c97-b410-eb15a38a1299 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:57.804919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4e6fe9-85b1-4c97-b410-eb15a38a1299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:57.822852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cf4e6fe9-85b1-4c97-b410-eb15a38a1299 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:23:57.824114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cf4e6fe9-85b1-4c97-b410-eb15a38a1299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:23:57.831853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 763/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:23:57.983443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b11eb3-a4b1-4ca0-9730-beade63cc02c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:23:57.997798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b11eb3-a4b1-4ca0-9730-beade63cc02c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:23:58.003133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f8f798-4094-4027-a64f-b4a6e2ecac47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:23:58.003793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f798-4094-4027-a64f-b4a6e2ecac47 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:58.004084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f798-4094-4027-a64f-b4a6e2ecac47 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:58.004363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f8f798-4094-4027-a64f-b4a6e2ecac47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:58.012403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8f798-4094-4027-a64f-b4a6e2ecac47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:23:58.014003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f8f798-4094-4027-a64f-b4a6e2ecac47 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:23:58.023288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 761/1524] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:23:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:23:58.834586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc06c-911c-4675-8209-06412a348528 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:58.835138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc06c-911c-4675-8209-06412a348528 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:58.835138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80bc06c-911c-4675-8209-06412a348528 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:58.836539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 764/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jul 7 10:23:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:23:58.841783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e4ac-9bb4-45fc-967c-670887af0e09 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:58.842223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e4ac-9bb4-45fc-967c-670887af0e09 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:58.842320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c636e4ac-9bb4-45fc-967c-670887af0e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:58.862013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c636e4ac-9bb4-45fc-967c-670887af0e09 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:23:58.863208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c636e4ac-9bb4-45fc-967c-670887af0e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:23:58.870839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 762/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jul 7 10:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:23:59.020159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bc06c-911c-4675-8209-06412a348528 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:23:59.034898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80bc06c-911c-4675-8209-06412a348528 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:23:59.040198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc79a3-49d0-4aeb-803b-4193217cc55a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:23:59.040832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc79a3-49d0-4aeb-803b-4193217cc55a service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:23:59.041114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc79a3-49d0-4aeb-803b-4193217cc55a service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:23:59.041389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc79a3-49d0-4aeb-803b-4193217cc55a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:23:59.049583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc79a3-49d0-4aeb-803b-4193217cc55a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:23:59.051153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc79a3-49d0-4aeb-803b-4193217cc55a service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:23:59.060409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 765/1527] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:24:03.090870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8da17-37f7-47fb-9c70-5f19285b9f7b None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:03.091441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8da17-37f7-47fb-9c70-5f19285b9f7b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:03.091441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b8da17-37f7-47fb-9c70-5f19285b9f7b None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:03.092837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 763/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:24:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:24:03.097899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c39f-d071-4299-8ae9-d1e86ae959e9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:03.098352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c39f-d071-4299-8ae9-d1e86ae959e9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:03.098463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30c39f-d071-4299-8ae9-d1e86ae959e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:03.116336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5f30c39f-d071-4299-8ae9-d1e86ae959e9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:24:03.117540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5f30c39f-d071-4299-8ae9-d1e86ae959e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:24:03.125884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 766/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:24:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:24:03.198117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8da17-37f7-47fb-9c70-5f19285b9f7b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:24:03.199820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-44b8da17-37f7-47fb-9c70-5f19285b9f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:24:03.255002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b8da17-37f7-47fb-9c70-5f19285b9f7b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:24:03.258492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1295d711-5db1-408c-b192-a1bcd83f0a98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:24:03.259134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295d711-5db1-408c-b192-a1bcd83f0a98 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:03.259980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295d711-5db1-408c-b192-a1bcd83f0a98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:03.260263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1295d711-5db1-408c-b192-a1bcd83f0a98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:03.268971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295d711-5db1-408c-b192-a1bcd83f0a98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:24:03.270385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1295d711-5db1-408c-b192-a1bcd83f0a98 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:24:03.275255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:24:03.280221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 764/1530] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:24:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:24:05.304512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bebf91-ea35-4dde-9a57-9b82d1e289dd None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:05.305060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bebf91-ea35-4dde-9a57-9b82d1e289dd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:05.305170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bebf91-ea35-4dde-9a57-9b82d1e289dd None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:05.306486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 767/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:24:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:24:05.311593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cb2b4-e89a-440a-a33c-5df09fdf5d2e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:05.312076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cb2b4-e89a-440a-a33c-5df09fdf5d2e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:05.312238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567cb2b4-e89a-440a-a33c-5df09fdf5d2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:05.330109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-567cb2b4-e89a-440a-a33c-5df09fdf5d2e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:24:05.331331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-567cb2b4-e89a-440a-a33c-5df09fdf5d2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:24:05.339592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 765/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:24:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:24:05.374943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bebf91-ea35-4dde-9a57-9b82d1e289dd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:24:05.385434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bebf91-ea35-4dde-9a57-9b82d1e289dd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:24:05.388866 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a9973-56da-4d60-b41a-e2f6e12e4c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:24:05.389493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a9973-56da-4d60-b41a-e2f6e12e4c95 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:05.389771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a9973-56da-4d60-b41a-e2f6e12e4c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:05.390043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a9973-56da-4d60-b41a-e2f6e12e4c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:05.398688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a9973-56da-4d60-b41a-e2f6e12e4c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:24:05.400679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a9973-56da-4d60-b41a-e2f6e12e4c95 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:24:05.409984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 768/1533] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:24:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:24:08.883669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee847c-3ead-413c-9c75-f7919ce7d8eb None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:08.884800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee847c-3ead-413c-9c75-f7919ce7d8eb None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:08.884800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcee847c-3ead-413c-9c75-f7919ce7d8eb None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:08.886185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 766/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:24:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:24:08.891305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826df1-9f40-45fd-9fba-e0741665dd3e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:08.891720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826df1-9f40-45fd-9fba-e0741665dd3e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:08.891901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f826df1-9f40-45fd-9fba-e0741665dd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:08.910049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8f826df1-9f40-45fd-9fba-e0741665dd3e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:24:08.911226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8f826df1-9f40-45fd-9fba-e0741665dd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:24:08.918933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 769/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:24:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:24:09.216568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee847c-3ead-413c-9c75-f7919ce7d8eb None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:24:09.218242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bcee847c-3ead-413c-9c75-f7919ce7d8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:24:09.273818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcee847c-3ead-413c-9c75-f7919ce7d8eb None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:24:09.277289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f7437b-d078-4b27-9abe-34d1fbc19d32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:24:09.277932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7437b-d078-4b27-9abe-34d1fbc19d32 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:09.278223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7437b-d078-4b27-9abe-34d1fbc19d32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:09.278503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f7437b-d078-4b27-9abe-34d1fbc19d32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:09.287122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7437b-d078-4b27-9abe-34d1fbc19d32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:24:09.288609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f7437b-d078-4b27-9abe-34d1fbc19d32 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:24:09.297983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 767/1536] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:24:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:24:11.325149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794748-b3c9-41d8-8709-0018800623d1 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:11.325769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794748-b3c9-41d8-8709-0018800623d1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:11.325769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e794748-b3c9-41d8-8709-0018800623d1 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:11.327070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 770/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jul 7 10:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:24:11.332143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75499f5e-45be-43c3-a0c7-1c04a8bca3bf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:11.332552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75499f5e-45be-43c3-a0c7-1c04a8bca3bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:11.332732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75499f5e-45be-43c3-a0c7-1c04a8bca3bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:11.351174 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-75499f5e-45be-43c3-a0c7-1c04a8bca3bf None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:24:11.352406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-75499f5e-45be-43c3-a0c7-1c04a8bca3bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:24:11.360883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 768/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jul 7 10:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:24:11.660041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e794748-b3c9-41d8-8709-0018800623d1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:24:11.661845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9e794748-b3c9-41d8-8709-0018800623d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:24:11.670092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=65 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:24:11.676621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e794748-b3c9-41d8-8709-0018800623d1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:24:11.680138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f2257-eeb9-48d3-b1aa-eea7f5056144 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:24:11.680777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2257-eeb9-48d3-b1aa-eea7f5056144 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:24:11.681082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2257-eeb9-48d3-b1aa-eea7f5056144 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:24:11.681368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2257-eeb9-48d3-b1aa-eea7f5056144 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:24:11.690178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f2257-eeb9-48d3-b1aa-eea7f5056144 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:24:11.691600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513f2257-eeb9-48d3-b1aa-eea7f5056144 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:24:11.701786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 771/1539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:24:21.345941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:24:21.697084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:42.794778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4054474-6f21-4e16-acdf-144fe6d53138 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:42.796212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4054474-6f21-4e16-acdf-144fe6d53138 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:42.796212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4054474-6f21-4e16-acdf-144fe6d53138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:42.818001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c4054474-6f21-4e16-acdf-144fe6d53138 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:42.819465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4054474-6f21-4e16-acdf-144fe6d53138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:42.834909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 769/1540] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:42.843497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a7dba-333c-4883-b8e6-7d02069eadcc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:42.844095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a7dba-333c-4883-b8e6-7d02069eadcc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:42.844095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4a7dba-333c-4883-b8e6-7d02069eadcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:42.873123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ca4a7dba-333c-4883-b8e6-7d02069eadcc None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:42.874364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ca4a7dba-333c-4883-b8e6-7d02069eadcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:42.890057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 772/1541] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:42.895148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4054474-6f21-4e16-acdf-144fe6d53138 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:42.905820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4054474-6f21-4e16-acdf-144fe6d53138 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:42.916164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:42.917057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:42.917171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:42.917402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:42.918149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:42.919012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:42.920685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:42.934626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 770/1542] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:42.942717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a7dba-333c-4883-b8e6-7d02069eadcc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:42.957683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4a7dba-333c-4883-b8e6-7d02069eadcc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:42.966671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a8ce9-ddd3-4cac-aee4-e0ced9469e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:42.967358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8ce9-ddd3-4cac-aee4-e0ced9469e87 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:42.967649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8ce9-ddd3-4cac-aee4-e0ced9469e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:42.967952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a8ce9-ddd3-4cac-aee4-e0ced9469e87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:42.968681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8ce9-ddd3-4cac-aee4-e0ced9469e87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:42.970643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a8ce9-ddd3-4cac-aee4-e0ced9469e87 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.021958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 773/1543] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:42 2022] POST /identity/v3/projects => generated 365 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:43.026206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.028729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663e673-bdd7-4f63-8ee3-d91a85ff37f8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.029027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663e673-bdd7-4f63-8ee3-d91a85ff37f8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.029319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6663e673-bdd7-4f63-8ee3-d91a85ff37f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.040635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87049b8c-b935-467e-ad49-65b1db5e60cd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.060464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-6663e673-bdd7-4f63-8ee3-d91a85ff37f8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.061713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6663e673-bdd7-4f63-8ee3-d91a85ff37f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.137545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.138243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.138533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.138812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.139522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.141501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.160507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 774/1544] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jul 07 10:25:43.171032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668da8c-95c9-4cc0-af52-23e47cb2266d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.171385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668da8c-95c9-4cc0-af52-23e47cb2266d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.171670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d668da8c-95c9-4cc0-af52-23e47cb2266d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.178873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 771/1545] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/users => generated 283 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:43.183103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.202942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d668da8c-95c9-4cc0-af52-23e47cb2266d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.203188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d417070-ae7e-45fa-9a2d-3063fa0c56a3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.205217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d668da8c-95c9-4cc0-af52-23e47cb2266d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.212987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.213700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.213997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.214320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.215032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.217469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.221697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 775/1546] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:25:43.228307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668da8c-95c9-4cc0-af52-23e47cb2266d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.240566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 772/1547] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:43.244051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.248014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d668da8c-95c9-4cc0-af52-23e47cb2266d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.258027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.259134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.259634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.260178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.261207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:43.261787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.261988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea3547b-bb89-4e98-b35b-f96d8f4e94d2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.263433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:25:43.263433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 07 10:25:43.263433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: Jul 07 10:25:43.264826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.285319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 776/1548] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:25:43.287190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.287889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.288181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.288457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.289197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.290963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.294258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.307195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 773/1549] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:43.310004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39abcb8-c0e2-42d1-9260-116f3fdf61ca admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.311344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.320377 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.321591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.322104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.322599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.323527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.323662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4018e8f-c13b-432c-ba9b-5e4421ab31f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.325876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.341278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c52cc-1b50-41cc-beb0-3a854f42f0cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.342200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c52cc-1b50-41cc-beb0-3a854f42f0cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.342488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c52cc-1b50-41cc-beb0-3a854f42f0cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.342770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c52cc-1b50-41cc-beb0-3a854f42f0cc admin admin] PATH_INFO: `/v3/projects/bf455075addb494f89fc180a2c0fc19f/users/25c62d6777c042baab53edee5556bdc3/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.369962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c52cc-1b50-41cc-beb0-3a854f42f0cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf455075addb494f89fc180a2c0fc19f, user_id=25c62d6777c042baab53edee5556bdc3, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.370578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 777/1550] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/projects => generated 373 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:25:43.371374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-ca6c52cc-1b50-41cc-beb0-3a854f42f0cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:25:43.371374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:25:43.371374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:25:43.373185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca6c52cc-1b50-41cc-beb0-3a854f42f0cc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.374761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.403465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 774/1551] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:43 2022] PUT /identity/v3/projects/bf455075addb494f89fc180a2c0fc19f/users/25c62d6777c042baab53edee5556bdc3/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:25:43.423780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ac0fe-58bd-4fbe-be4e-a3ad86a2f1c3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.423780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ac0fe-58bd-4fbe-be4e-a3ad86a2f1c3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.423780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ac0fe-58bd-4fbe-be4e-a3ad86a2f1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.466418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d92ac0fe-58bd-4fbe-be4e-a3ad86a2f1c3 None None] MFA Rules not processed for user `25c62d6777c042baab53edee5556bdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.467640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d92ac0fe-58bd-4fbe-be4e-a3ad86a2f1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.469966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3f4bfe-c7e4-4b4f-bd41-e6f06e13bfb1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.556124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:43.612111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54dfdcf-9300-4071-b937-8afaf6173f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.613676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dfdcf-9300-4071-b937-8afaf6173f42 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.613962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dfdcf-9300-4071-b937-8afaf6173f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.615997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dfdcf-9300-4071-b937-8afaf6173f42 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.616728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54dfdcf-9300-4071-b937-8afaf6173f42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.617774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-f54dfdcf-9300-4071-b937-8afaf6173f42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:25:43.617774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 07 10:25:43.617774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: Jul 07 10:25:43.619009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54dfdcf-9300-4071-b937-8afaf6173f42 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.634053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 775/1552] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:25:43.635186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d896aee-354c-44e2-91be-85421eb19d6b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.635482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d896aee-354c-44e2-91be-85421eb19d6b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.635778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d896aee-354c-44e2-91be-85421eb19d6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.664198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 778/1553] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/users => generated 287 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:25:43.664424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0d896aee-354c-44e2-91be-85421eb19d6b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.665330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0d896aee-354c-44e2-91be-85421eb19d6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.665464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13bdd-26e6-4450-8f70-c74e6fc04bca None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.665754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13bdd-26e6-4450-8f70-c74e6fc04bca None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.666030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb13bdd-26e6-4450-8f70-c74e6fc04bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.673904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 776/1554] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.675145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef5cc7-5a2c-4b3a-8dff-7adf846d16df None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.675441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef5cc7-5a2c-4b3a-8dff-7adf846d16df None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.675725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aef5cc7-5a2c-4b3a-8dff-7adf846d16df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.688944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:43.691540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ffb13bdd-26e6-4450-8f70-c74e6fc04bca None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.693038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ffb13bdd-26e6-4450-8f70-c74e6fc04bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.696470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6aef5cc7-5a2c-4b3a-8dff-7adf846d16df None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.697659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6aef5cc7-5a2c-4b3a-8dff-7adf846d16df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.706425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 779/1555] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.706839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 777/1556] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.707451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb13bdd-26e6-4450-8f70-c74e6fc04bca None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.708238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f99dd-939b-41db-ba11-a14ee8ab87b8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.708533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f99dd-939b-41db-ba11-a14ee8ab87b8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.708863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9f99dd-939b-41db-ba11-a14ee8ab87b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.719771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb13bdd-26e6-4450-8f70-c74e6fc04bca None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.724561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.725247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.725531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.725808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.726495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.728444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.729936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-de9f99dd-939b-41db-ba11-a14ee8ab87b8 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.731206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-de9f99dd-939b-41db-ba11-a14ee8ab87b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.739411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 778/1557] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.740784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a2a4-10be-4dc6-823b-9d88d8b09403 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.741078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a2a4-10be-4dc6-823b-9d88d8b09403 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.741361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d14a2a4-10be-4dc6-823b-9d88d8b09403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.755155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 780/1558] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:43.756161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.759687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7d14a2a4-10be-4dc6-823b-9d88d8b09403 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.760944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7d14a2a4-10be-4dc6-823b-9d88d8b09403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.818085 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2323f242-3fcc-4b29-b94a-f3068cae9c24 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.819922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 779/1559] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.821275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52631f-2d5f-4f9a-93ec-f6f90bc38905 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.821576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52631f-2d5f-4f9a-93ec-f6f90bc38905 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.821862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c52631f-2d5f-4f9a-93ec-f6f90bc38905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.828772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957545b3-76d0-48ec-8d15-fa147430460a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.829441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957545b3-76d0-48ec-8d15-fa147430460a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.829720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957545b3-76d0-48ec-8d15-fa147430460a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.829997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957545b3-76d0-48ec-8d15-fa147430460a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.830753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957545b3-76d0-48ec-8d15-fa147430460a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.832427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957545b3-76d0-48ec-8d15-fa147430460a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.845860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 781/1560] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:43.846542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7c52631f-2d5f-4f9a-93ec-f6f90bc38905 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.846678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c703563-aaf2-4197-886d-675216ffde36 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.846678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c703563-aaf2-4197-886d-675216ffde36 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.846855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7c52631f-2d5f-4f9a-93ec-f6f90bc38905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.846953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c703563-aaf2-4197-886d-675216ffde36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.853468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 780/1561] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.854696 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe221095-f97b-4ef4-a0e9-a9ed31133551 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.854989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe221095-f97b-4ef4-a0e9-a9ed31133551 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.855325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe221095-f97b-4ef4-a0e9-a9ed31133551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.864002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8c703563-aaf2-4197-886d-675216ffde36 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.865189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8c703563-aaf2-4197-886d-675216ffde36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.873107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fe221095-f97b-4ef4-a0e9-a9ed31133551 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.873530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 782/1562] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.874275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fe221095-f97b-4ef4-a0e9-a9ed31133551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.874715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be0a9e-8440-485e-9e58-4f446605d0cd None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.875000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be0a9e-8440-485e-9e58-4f446605d0cd None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.875274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34be0a9e-8440-485e-9e58-4f446605d0cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.886016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 781/1563] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.887935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689542-2ccc-4497-87db-ab3158a3919c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.888228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689542-2ccc-4497-87db-ab3158a3919c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.888534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93689542-2ccc-4497-87db-ab3158a3919c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.896693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-34be0a9e-8440-485e-9e58-4f446605d0cd None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.897917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-34be0a9e-8440-485e-9e58-4f446605d0cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.905628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-93689542-2ccc-4497-87db-ab3158a3919c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.906087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 783/1564] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.906790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-93689542-2ccc-4497-87db-ab3158a3919c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.906902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be0a9e-8440-485e-9e58-4f446605d0cd None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.914997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 782/1565] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.916285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c63ab2-f2d0-4db9-9943-b251fd011472 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.916412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34be0a9e-8440-485e-9e58-4f446605d0cd None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.916739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c63ab2-f2d0-4db9-9943-b251fd011472 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.917037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c63ab2-f2d0-4db9-9943-b251fd011472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.920984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.921635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.921916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.922191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.922871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.924828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.935332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-11c63ab2-f2d0-4db9-9943-b251fd011472 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.936709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-11c63ab2-f2d0-4db9-9943-b251fd011472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:43.945414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 783/1566] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:43.949232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c63ab2-f2d0-4db9-9943-b251fd011472 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.949382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 784/1567] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/users => generated 290 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:43.949382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.958776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c63ab2-f2d0-4db9-9943-b251fd011472 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.959711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f00a7f-05e6-4c46-99ad-29a745bf43b1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.963377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c7c19-72cc-437b-aeea-b3a81aa39308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.964079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c7c19-72cc-437b-aeea-b3a81aa39308 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.964252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.964380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c7c19-72cc-437b-aeea-b3a81aa39308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.964629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c7c19-72cc-437b-aeea-b3a81aa39308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.964891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.965203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.965379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c7c19-72cc-437b-aeea-b3a81aa39308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.965488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.966195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:43.966503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.966993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c7c19-72cc-437b-aeea-b3a81aa39308 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.968197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:43.977219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 784/1568] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:43.978709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd64ed-54a1-4c1d-a9ef-6e11bc994f8e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.979068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd64ed-54a1-4c1d-a9ef-6e11bc994f8e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.979423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd64ed-54a1-4c1d-a9ef-6e11bc994f8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.981156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 785/1569] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:43.981156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:43.992067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8989b927-8f86-43b1-b3da-b77f1bc6a0df admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:43.997291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:43.997934 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:43.998219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:43.998497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:43.998651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-22dd64ed-54a1-4c1d-a9ef-6e11bc994f8e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:43.999237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:43.999548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:43.999873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-22dd64ed-54a1-4c1d-a9ef-6e11bc994f8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:44.001252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.007900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 785/1570] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:44.010903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd64ed-54a1-4c1d-a9ef-6e11bc994f8e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.021094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 786/1571] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:44.021094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.021467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd64ed-54a1-4c1d-a9ef-6e11bc994f8e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.025010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea06a97-fc53-4c5b-9f07-a63f650bc459 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.026153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.026811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.027093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.027371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.028162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:44.028471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.029604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.030055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.030241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.030510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.030789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.031611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:44.032054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.033627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.041805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 786/1572] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:44.042763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.051885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 787/1573] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:44.052697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f32a38-9fb9-418f-88ce-602512a3e1cc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.052911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.057066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.057709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.057994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.058270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.059522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:44.059861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.061441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.063224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6406cb2e-89c3-4508-bcbe-dbf1f65d6d06 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.067823 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.068470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.068753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.069028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.069992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.071840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.075311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 787/1574] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:44.076183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.082471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 788/1575] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.083127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.088692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df6a1b-c968-419e-bf67-2bf9745325f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.093067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca98858-3464-4d7d-b88e-76bdeb7be92f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.093423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.094312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.094606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.094890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] PATH_INFO: `/v3/projects/98924fde961c41358ffb24ac3819557a/users/247787cd745444f38953a64780d6ac9b/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.097825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.098433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.098720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.099000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.099857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.101893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.109675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98924fde961c41358ffb24ac3819557a, user_id=247787cd745444f38953a64780d6ac9b, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.112323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:44.112478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.127671 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 788/1576] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:44 2022] PUT /identity/v3/projects/98924fde961c41358ffb24ac3819557a/users/247787cd745444f38953a64780d6ac9b/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:44.128518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.129157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 789/1577] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.129955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.191764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef5eea-06fa-4b55-9400-8bafe3f3267b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.192339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb65daa9-2079-438a-a83a-d8161bbb3f45 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.196645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.197299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.197454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.197581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.197848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.198112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.198401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.198534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.198680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.199404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:44.200281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.201070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.201879 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.203383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:44.215345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 789/1578] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:44.216210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.226014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe16e5b-5062-4ce2-83c0-e2ffda6b9528 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.231343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.232025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.232311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.232586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.233271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.234275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:25:44.234275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:25:44.234275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:25:44.238240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 790/1579] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/projects => generated 369 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.238389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.238489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.264500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 790/1580] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/projects => generated 393 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.265548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.300481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf57c43-c7ac-4dd5-ab3c-3da6278750e7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.327035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81490b88-5b15-49dd-9f81-282f2d5addb5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.331869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.332537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.332928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.333221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.333915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.335906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.357440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9433f3-13dd-4acd-8f0e-eb26d7de9742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.358028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9433f3-13dd-4acd-8f0e-eb26d7de9742 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.358316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9433f3-13dd-4acd-8f0e-eb26d7de9742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.358593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9433f3-13dd-4acd-8f0e-eb26d7de9742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.359307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9433f3-13dd-4acd-8f0e-eb26d7de9742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.361455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9433f3-13dd-4acd-8f0e-eb26d7de9742 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.361922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 791/1581] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/projects => generated 405 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.362599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.391553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 791/1582] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/projects => generated 379 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.391553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019614a1-4f28-4600-a201-2746bd57f346 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.391553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019614a1-4f28-4600-a201-2746bd57f346 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.391553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019614a1-4f28-4600-a201-2746bd57f346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.407216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-019614a1-4f28-4600-a201-2746bd57f346 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:44.408550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-019614a1-4f28-4600-a201-2746bd57f346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:44.412558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:44.426490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7285e97-f2b0-431c-9369-a7635c2eaa49 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.469251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 792/1583] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:44.470235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019614a1-4f28-4600-a201-2746bd57f346 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.479867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019614a1-4f28-4600-a201-2746bd57f346 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.485003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.485658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.485941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.486217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.486932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.487254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.488200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.488490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.488767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] PATH_INFO: `/v3/projects/1a6c8ef820104ed09e026d7c92ec5a26/users/9e420abe8b864523bdf21fbddde8677d/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.488891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.497222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:44.506509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6c8ef820104ed09e026d7c92ec5a26, user_id=9e420abe8b864523bdf21fbddde8677d, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.510116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.511877 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 793/1584] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/users => generated 287 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.512733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.522933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb98d76b-d29b-4fe1-bd1b-5d23fe9d7803 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.527243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 792/1585] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:44 2022] PUT /identity/v3/projects/1a6c8ef820104ed09e026d7c92ec5a26/users/9e420abe8b864523bdf21fbddde8677d/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:44.530339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.578073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.578733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.579042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.579316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.580023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.581922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.589234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071e7406-fff8-4b89-880a-a384933cd08b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.594582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.595404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.596176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.596595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.597299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.599500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.607571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 794/1586] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/projects => generated 377 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.608394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.624274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 793/1587] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.624274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.633814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b33c60-f6f8-42dd-8c9f-01874e615038 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.670165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb08d6c1-90c9-4407-a5f1-df3c3fe1896c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.674710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.675379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.675652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.675963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.676650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.678574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.689750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.690660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.691041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.691408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.692257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.694683 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.700715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 795/1588] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.701414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.711087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53410aaf-9e08-4923-884c-30d4312cfd92 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.718203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 794/1589] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.718203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.719080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.719760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.720100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.720362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.721061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.722984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.729621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:44.745527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 796/1590] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/users => generated 290 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.746412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.755771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516e0ad3-018f-4640-891b-98eabbb59c49 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.760345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.760984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.761263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.761533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.762269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.763881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.774456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 797/1591] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.774815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f4517d-cc87-4036-b05e-a0dbf8618558 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.775597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.778942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.779594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.779947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.780231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.785886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae23bdb7-481e-43fc-b6d4-0cd633ed09dc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.789510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.790459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.790939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.791086 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.791370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.791647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.792377 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.794496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.801200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 795/1592] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:44.801989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.804903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 798/1593] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.808706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.811365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f49bde-48ab-423a-87d4-3a2b3ce6e2ec service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.816279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.817147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.817407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.817717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.818413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.819580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9cdb01-809d-4d2c-ba08-5b306ec886e5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.820732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.825134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.825829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.826124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.826465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.827262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.829100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.831050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:44.845828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 799/1594] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.845828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.849793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 796/1595] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:44 2022] POST /identity/v3/users => generated 289 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:44.850812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.860163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04382ea6-f862-4cde-a9a9-6dea434da378 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.862230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88f41bd-e173-4c17-9330-ae36c68066f1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.865482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.866181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.866528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.866740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.867462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.867935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.868689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.869062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.869191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.869741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.870590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.872442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.880273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 800/1596] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.884178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.885118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 797/1597] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.885837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.895489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e441985-51dc-400f-80ea-954e910c3b57 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.898094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d770ae-abb6-4b30-899e-cc6b61f579fe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.901079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.901745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.902029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.902307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.903687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.904048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.904775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.905132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.905340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.905545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.906329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.908006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.916059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 801/1598] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.916936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.926132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 798/1599] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.926132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.931128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6206d70c-8bb8-49a8-ba31-cb0548ca75b1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.936389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.936911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af554e6f-26f7-4c22-a84f-9b4ddce6f92d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:44.937673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.938020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.938325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] PATH_INFO: `/v3/projects/1a6c8ef820104ed09e026d7c92ec5a26/users/9e420abe8b864523bdf21fbddde8677d/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.940984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a6c8ef820104ed09e026d7c92ec5a26, user_id=9e420abe8b864523bdf21fbddde8677d, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.942267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:44.942704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:44.943059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:44.943407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:44.944858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:44.944976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.946192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:44.964681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 802/1600] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:44 2022] PUT /identity/v3/projects/1a6c8ef820104ed09e026d7c92ec5a26/users/9e420abe8b864523bdf21fbddde8677d/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:44.964681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.965771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 799/1601] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:44.965771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:44.972824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198bb6d-9412-4296-b0ed-3296e9668d2a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.034485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68f9a71-34fb-45bb-a607-74577420ced9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.036803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.037486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.037776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.038058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.038792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.040078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.040449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.040960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.041395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.041835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.042952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.045190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.054537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 800/1602] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:45.055246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.065673 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff0f9ce-f093-4b72-8936-00efaf6a6ef5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.072160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 803/1603] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:44 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:45.073120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.073276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.073818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.074255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.074439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.076050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.077726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.085179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637632b6-a16b-4589-a55f-89574a72e0a0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.089553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 801/1604] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:45.090971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.096401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e424c3f8-56ad-4fa3-987a-62b88f9385b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.097559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c3f8-56ad-4fa3-987a-62b88f9385b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.097559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c3f8-56ad-4fa3-987a-62b88f9385b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.097786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e424c3f8-56ad-4fa3-987a-62b88f9385b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.098352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424c3f8-56ad-4fa3-987a-62b88f9385b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.099997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e424c3f8-56ad-4fa3-987a-62b88f9385b4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.102137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad82332-02ac-4dcd-8364-e470505e1038 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.107316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.108645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.109015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.109391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] PATH_INFO: `/v3/projects/2c6a404439ce43e4b6210fcf354a5472/users/b41ed92aeada435ea82e26acc6016578/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.120689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 804/1605] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:45.122140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a61b-d898-4390-b4f5-b85791b98011 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.122427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a61b-d898-4390-b4f5-b85791b98011 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.122784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e33a61b-d898-4390-b4f5-b85791b98011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.133486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c6a404439ce43e4b6210fcf354a5472, user_id=b41ed92aeada435ea82e26acc6016578, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.144583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.158016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4e33a61b-d898-4390-b4f5-b85791b98011 None None] MFA Rules not processed for user `9e420abe8b864523bdf21fbddde8677d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:45.159351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e33a61b-d898-4390-b4f5-b85791b98011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:45.173558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 802/1606] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:45 2022] PUT /identity/v3/projects/2c6a404439ce43e4b6210fcf354a5472/users/b41ed92aeada435ea82e26acc6016578/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:45.173558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.243145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:45.257002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173722c-5767-45ac-bac9-7a775444fb91 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.264196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.265072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 805/1607] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:45.265452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.265867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.265988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a61b-d898-4390-b4f5-b85791b98011 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.266398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] PATH_INFO: `/v3/projects/c8f89082d97f4967910ab5dccc941265/users/973c27dbce9a40e59acd7753286f1cb6/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.283489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e33a61b-d898-4390-b4f5-b85791b98011 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.289209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.289708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8f89082d97f4967910ab5dccc941265, user_id=973c27dbce9a40e59acd7753286f1cb6, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.290108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.290387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.290664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] PATH_INFO: `/v3/projects/0a883b49db0f4f32a32bf6d87e2f6104/users/92b758b22f054dd991e8af99354a134f/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.293156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.308367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a883b49db0f4f32a32bf6d87e2f6104, user_id=92b758b22f054dd991e8af99354a134f, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.316054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 803/1608] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:45 2022] PUT /identity/v3/projects/c8f89082d97f4967910ab5dccc941265/users/973c27dbce9a40e59acd7753286f1cb6/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:45.316233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.316350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.325568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:45.331707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 806/1609] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:45 2022] PUT /identity/v3/projects/0a883b49db0f4f32a32bf6d87e2f6104/users/92b758b22f054dd991e8af99354a134f/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:45.332674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.397930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da4bd5e-0ab0-4c39-bebd-a105fcc00dd4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.399638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe4aeae-bf1f-4874-b752-0e77d5f7a61c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.402833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405f6e34-321a-4722-ac2f-9a46aef269b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.404267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f6e34-321a-4722-ac2f-9a46aef269b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.404625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f6e34-321a-4722-ac2f-9a46aef269b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.404967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405f6e34-321a-4722-ac2f-9a46aef269b3 admin admin] PATH_INFO: `/v3/projects/26866bd48a1b436e983df989ea51bcbe/users/32d8a8e3820348d9885b8447104fb72c/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.405195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.405948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.406300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.406655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.407472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.409315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.424446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f6e34-321a-4722-ac2f-9a46aef269b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26866bd48a1b436e983df989ea51bcbe, user_id=32d8a8e3820348d9885b8447104fb72c, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.428224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 804/1610] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:45 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:45.428224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.428462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405f6e34-321a-4722-ac2f-9a46aef269b3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.438605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41e3828-5b27-4875-8d5f-8301e3276271 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.442337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 807/1611] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:45 2022] PUT /identity/v3/projects/26866bd48a1b436e983df989ea51bcbe/users/32d8a8e3820348d9885b8447104fb72c/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:45.446221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50816b4e-ecf2-4b11-8ddf-9d51f58c55d8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.446221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50816b4e-ecf2-4b11-8ddf-9d51f58c55d8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.446221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50816b4e-ecf2-4b11-8ddf-9d51f58c55d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.472308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-50816b4e-ecf2-4b11-8ddf-9d51f58c55d8 None None] MFA Rules not processed for user `b41ed92aeada435ea82e26acc6016578`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:45.473553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-50816b4e-ecf2-4b11-8ddf-9d51f58c55d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:45.513370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c573cd9-6a3c-4d1a-b4a6-f7b6d542cd56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.515058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c573cd9-6a3c-4d1a-b4a6-f7b6d542cd56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.515579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c573cd9-6a3c-4d1a-b4a6-f7b6d542cd56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.516316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c573cd9-6a3c-4d1a-b4a6-f7b6d542cd56 admin admin] PATH_INFO: `/v3/projects/7d14c724ddb346539e903734794ccf42/users/e8287678ddfb46d5a89f23a9c6f3dc00/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.534170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c573cd9-6a3c-4d1a-b4a6-f7b6d542cd56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d14c724ddb346539e903734794ccf42, user_id=e8287678ddfb46d5a89f23a9c6f3dc00, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.537325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c573cd9-6a3c-4d1a-b4a6-f7b6d542cd56 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.548879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 808/1612] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:45.550338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50816b4e-ecf2-4b11-8ddf-9d51f58c55d8 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.556623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 805/1613] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:45 2022] PUT /identity/v3/projects/7d14c724ddb346539e903734794ccf42/users/e8287678ddfb46d5a89f23a9c6f3dc00/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:45.558038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9661-ef31-432c-91f6-c7a49ff139f9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.558414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9661-ef31-432c-91f6-c7a49ff139f9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.558698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ea9661-ef31-432c-91f6-c7a49ff139f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.596271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-71ea9661-ef31-432c-91f6-c7a49ff139f9 None None] MFA Rules not processed for user `973c27dbce9a40e59acd7753286f1cb6`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:45.598017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71ea9661-ef31-432c-91f6-c7a49ff139f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:45.620704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:45.648808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50816b4e-ecf2-4b11-8ddf-9d51f58c55d8 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.695666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 806/1614] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:45.697222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2792-f1d3-4d99-bfda-100e34ef6f27 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.697529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2792-f1d3-4d99-bfda-100e34ef6f27 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.697807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168f2792-f1d3-4d99-bfda-100e34ef6f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.712586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.713577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.714238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.714387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.730101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-168f2792-f1d3-4d99-bfda-100e34ef6f27 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:45.732356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-168f2792-f1d3-4d99-bfda-100e34ef6f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:45.739817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:45.807593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.809839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.821353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 809/1615] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:45 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:45.822409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.839825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 807/1616] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:45.839825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4895b5f-1cd3-4ec8-b4db-1a9efe3acc07 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.839825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4895b5f-1cd3-4ec8-b4db-1a9efe3acc07 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.839825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4895b5f-1cd3-4ec8-b4db-1a9efe3acc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.864732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e4895b5f-1cd3-4ec8-b4db-1a9efe3acc07 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:45.866647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e4895b5f-1cd3-4ec8-b4db-1a9efe3acc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:45.878301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 808/1617] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:45.879808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.880200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.880584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.907779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5d038f-f8aa-4df3-bc64-b9a6e5555342 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:45.913929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb1cc-522a-45dc-ac9c-668c09484443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:45.914991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb1cc-522a-45dc-ac9c-668c09484443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.915383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb1cc-522a-45dc-ac9c-668c09484443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.915730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb1cc-522a-45dc-ac9c-668c09484443 admin admin] PATH_INFO: `/v3/projects/0c765bc215964bce8a321b4f3d085ae6/users/730e7d9b92c84bd5872bc0e221b616cf/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.917551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] MFA Rules not processed for user `e8287678ddfb46d5a89f23a9c6f3dc00`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:45.919290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:45.938028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cb1cc-522a-45dc-ac9c-668c09484443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c765bc215964bce8a321b4f3d085ae6, user_id=730e7d9b92c84bd5872bc0e221b616cf, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:45.941217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cb1cc-522a-45dc-ac9c-668c09484443 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:45.967695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 810/1618] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:45 2022] PUT /identity/v3/projects/0c765bc215964bce8a321b4f3d085ae6/users/730e7d9b92c84bd5872bc0e221b616cf/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:45.973050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b05ed-ad7e-4bfa-8f5e-b231676d8e2e None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.973158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b05ed-ad7e-4bfa-8f5e-b231676d8e2e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.973402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b05ed-ad7e-4bfa-8f5e-b231676d8e2e None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.974764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 811/1619] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jul 7 10:25:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:25:45.982121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d4ba9-a7ef-44e5-9fd3-e1e5e7161518 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:45.982452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d4ba9-a7ef-44e5-9fd3-e1e5e7161518 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:45.982675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88d4ba9-a7ef-44e5-9fd3-e1e5e7161518 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:45.984187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 812/1620] 10.222.0.16 () {54 vars in 961 bytes} [Thu Jul 7 10:25:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:25:45.989114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d4ba9-a7ef-44e5-9fd3-e1e5e7161518 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:45.991001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f88d4ba9-a7ef-44e5-9fd3-e1e5e7161518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:46.031396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:46.045026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 809/1621] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:45 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:46.046048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.047487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:46.082621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88d4ba9-a7ef-44e5-9fd3-e1e5e7161518 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.087378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3793271-9d5e-4e44-8422-49c6d4649022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.088405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3793271-9d5e-4e44-8422-49c6d4649022 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.088968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3793271-9d5e-4e44-8422-49c6d4649022 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.089573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3793271-9d5e-4e44-8422-49c6d4649022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.133362 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d8df62-e706-42a6-8adb-9412446b5b2c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.140063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.141397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.141827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.142207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.177374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3793271-9d5e-4e44-8422-49c6d4649022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.178868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3793271-9d5e-4e44-8422-49c6d4649022 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.194814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 813/1622] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:46.196334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efed9d-4916-4b1e-a38c-b860ce00538d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.196764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efed9d-4916-4b1e-a38c-b860ce00538d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.197104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30efed9d-4916-4b1e-a38c-b860ce00538d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.210851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:46.224401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-30efed9d-4916-4b1e-a38c-b860ce00538d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:46.225975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-30efed9d-4916-4b1e-a38c-b860ce00538d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:46.227312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.228523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.239547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 810/1623] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:46.243185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.249864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 814/1624] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:46.251453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6036c43-be07-404d-a381-a2f3611f6d8c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.251971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6036c43-be07-404d-a381-a2f3611f6d8c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.252689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6036c43-be07-404d-a381-a2f3611f6d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.278172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e6036c43-be07-404d-a381-a2f3611f6d8c None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:46.279933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6036c43-be07-404d-a381-a2f3611f6d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:46.306608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0533823a-6103-41ca-9c83-6f092e6fc623 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.312253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.312891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.313178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.313457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.415520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.417346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.429346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 815/1626] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jul 7 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:46.431114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45f3f-271d-4ffe-8a1a-f67685d10085 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.431570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45f3f-271d-4ffe-8a1a-f67685d10085 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.432018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f45f3f-271d-4ffe-8a1a-f67685d10085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.437282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 811/1626] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:46.438539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.454980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d084ad6-d93f-477e-8a2a-08b712817b73 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.461322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.462057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.462469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.462754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.463606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.465706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.470976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-89f45f3f-271d-4ffe-8a1a-f67685d10085 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:46.472210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-89f45f3f-271d-4ffe-8a1a-f67685d10085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:46.494211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 816/1627] 10.222.0.16 () {58 vars in 1058 bytes} [Thu Jul 7 10:25:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:46.498060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45f3f-271d-4ffe-8a1a-f67685d10085 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.504900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 812/1628] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:46 2022] POST /identity/v3/projects => generated 377 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:46.505929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.523966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f45f3f-271d-4ffe-8a1a-f67685d10085 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.622965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d4779-a9bd-42be-b2a0-cd31b11ec3c1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.629226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.630133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.630609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.630984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.633346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.634501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.635009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.635645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.646445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:46.652712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:46.727573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.727573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.748872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 813/1629] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:46.751632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.759641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.763405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.765331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfb44d7-a0d9-48ff-8f48-8bc76c2cfbfd service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.771497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.773578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.773866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.774143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.774948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.777109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.800627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 817/1630] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:46.803889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.811011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 814/1631] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:46 2022] POST /identity/v3/users => generated 295 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:46.816222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.839364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b96b77-bc60-480e-a02a-9bf84ae7b8e8 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.841456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23c09a-c60c-4bde-a0f8-b35788c7be5d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.851042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.852700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.853654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.853940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.854212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.854754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.855035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.855299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:46.856614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:46.859795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:46.887962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 815/1632] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:46 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:46.889158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:46.914520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72faf0d2-d052-4ecd-b245-f56869b401f3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:46.985029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:46.985029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:46.985029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:46.985029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] PATH_INFO: `/v3/projects/5751156d7da241b486bad5d9434133a5/users/a19558c8991f41aeb2b910cc27905f31/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.049568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5751156d7da241b486bad5d9434133a5, user_id=a19558c8991f41aeb2b910cc27905f31, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:47.052392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:47.108878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:47.117523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:47.125590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 816/1633] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:46 2022] PUT /identity/v3/projects/5751156d7da241b486bad5d9434133a5/users/a19558c8991f41aeb2b910cc27905f31/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:47.126611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:47.158060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 818/1634] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:46 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:47.274054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf49163-2c23-4fa1-9951-510cd0ce6981 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:47.301905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093b632d-1f3a-4803-b84b-708216f12c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:47.305541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b632d-1f3a-4803-b84b-708216f12c7c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.308178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b632d-1f3a-4803-b84b-708216f12c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.308497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093b632d-1f3a-4803-b84b-708216f12c7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.315704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b632d-1f3a-4803-b84b-708216f12c7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:47.317676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093b632d-1f3a-4803-b84b-708216f12c7c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:47.349328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 817/1635] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:47.358689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:47.374827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1d75c-3b40-467f-adfa-bbdbdc9943a7 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:47.384442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313728e1-8e9b-42bc-a98b-588dff2d1d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:47.385559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313728e1-8e9b-42bc-a98b-588dff2d1d5a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.385934 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313728e1-8e9b-42bc-a98b-588dff2d1d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.386344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313728e1-8e9b-42bc-a98b-588dff2d1d5a admin admin] PATH_INFO: `/v3/projects/5751156d7da241b486bad5d9434133a5/users/a19558c8991f41aeb2b910cc27905f31/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.389572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313728e1-8e9b-42bc-a98b-588dff2d1d5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5751156d7da241b486bad5d9434133a5, user_id=a19558c8991f41aeb2b910cc27905f31, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:47.397670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313728e1-8e9b-42bc-a98b-588dff2d1d5a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:47.423983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 819/1636] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:47 2022] PUT /identity/v3/projects/5751156d7da241b486bad5d9434133a5/users/a19558c8991f41aeb2b910cc27905f31/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:47.426118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09c1e-2dad-4b42-8f73-0f858d704d08 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.426420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09c1e-2dad-4b42-8f73-0f858d704d08 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.426707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c09c1e-2dad-4b42-8f73-0f858d704d08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.433304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766f0b-1c2a-48d4-a21e-d029c45ee34d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.446099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766f0b-1c2a-48d4-a21e-d029c45ee34d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.446631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54766f0b-1c2a-48d4-a21e-d029c45ee34d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.448676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 820/1637] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jul 7 10:25:47 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:25:47.457672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edc5f9-eab7-4cfe-a54e-78e3d221895e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.459588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edc5f9-eab7-4cfe-a54e-78e3d221895e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.460072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57edc5f9-eab7-4cfe-a54e-78e3d221895e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.473919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-20c09c1e-2dad-4b42-8f73-0f858d704d08 None None] MFA Rules not processed for user `a19558c8991f41aeb2b910cc27905f31`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:47.475262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-20c09c1e-2dad-4b42-8f73-0f858d704d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:47.511792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-57edc5f9-eab7-4cfe-a54e-78e3d221895e None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:47.524721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-57edc5f9-eab7-4cfe-a54e-78e3d221895e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:47.543370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:47.622871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 818/1638] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:47 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:47.639167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c09c1e-2dad-4b42-8f73-0f858d704d08 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:47.712671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 821/1639] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:25:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:47.835968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c09c1e-2dad-4b42-8f73-0f858d704d08 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:47.849174 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10604011-7a8f-4b69-b5a0-3b7fb6bb1378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:47.849890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604011-7a8f-4b69-b5a0-3b7fb6bb1378 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.850149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604011-7a8f-4b69-b5a0-3b7fb6bb1378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.850432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10604011-7a8f-4b69-b5a0-3b7fb6bb1378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.898766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edc5f9-eab7-4cfe-a54e-78e3d221895e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:47.899768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10604011-7a8f-4b69-b5a0-3b7fb6bb1378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:47.902292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10604011-7a8f-4b69-b5a0-3b7fb6bb1378 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:47.910776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57edc5f9-eab7-4cfe-a54e-78e3d221895e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:47.929937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:47.930608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.930898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.931178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.939563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 819/1640] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:47 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:47.946249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae444df-b9e8-45c5-a77f-caee9863d010 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.946898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae444df-b9e8-45c5-a77f-caee9863d010 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.947399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae444df-b9e8-45c5-a77f-caee9863d010 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.952945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 820/1641] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jul 7 10:25:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:25:47.955409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:47.956916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:47.957712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fdc39-53f1-423b-b9b4-b65372e32f39 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:47.958236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fdc39-53f1-423b-b9b4-b65372e32f39 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:47.958723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46fdc39-53f1-423b-b9b4-b65372e32f39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:47.986257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b46fdc39-53f1-423b-b9b4-b65372e32f39 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:47.988403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b46fdc39-53f1-423b-b9b4-b65372e32f39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:48.007639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 822/1642] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:25:48.008630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 821/1643] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:25:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:48.084169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:48.092006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fdc39-53f1-423b-b9b4-b65372e32f39 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:48.100955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159e0943-57ac-4060-9b6e-9e79de99bc5a service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:48.106926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46fdc39-53f1-423b-b9b4-b65372e32f39 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:48.113226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:48.114635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:48.115320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:48.116018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:48.116986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edbaaa-e7d5-4d56-9a77-ddf9eff02352 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:48.119539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edbaaa-e7d5-4d56-9a77-ddf9eff02352 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:48.120242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edbaaa-e7d5-4d56-9a77-ddf9eff02352 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:48.120500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edbaaa-e7d5-4d56-9a77-ddf9eff02352 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:48.158201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edbaaa-e7d5-4d56-9a77-ddf9eff02352 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:48.160124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1edbaaa-e7d5-4d56-9a77-ddf9eff02352 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:48.177705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:48.188104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 823/1644] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:48.206003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:48.208408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:48.236908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 822/1645] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:48 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:25:48.470358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce725b-d374-4f2e-87ec-938f93d5ca53 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:48.470654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce725b-d374-4f2e-87ec-938f93d5ca53 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:48.470936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ce725b-d374-4f2e-87ec-938f93d5ca53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:48.537682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-04ce725b-d374-4f2e-87ec-938f93d5ca53 None None] MFA Rules not processed for user `32d8a8e3820348d9885b8447104fb72c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:48.538931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-04ce725b-d374-4f2e-87ec-938f93d5ca53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:48.667344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 824/1646] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:48 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:48.673531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:48.823627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce725b-d374-4f2e-87ec-938f93d5ca53 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:48.924213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d39eb93-f971-4381-ba92-abecd7f6009b service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:48.938753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:48.939456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:48.939785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:48.940046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:48.940752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:48.945205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:49.014517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 823/1647] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:48 2022] POST /identity/v3/projects => generated 405 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:25:49.018402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:49.019115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ce725b-d374-4f2e-87ec-938f93d5ca53 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:49.024795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:49.176895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53339b7-d192-48e5-875c-65b015f51bdd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:49.197776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:49.198618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:49.198886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:49.200865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:49.201627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:49.207699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:49.218709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:49.220640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:49.222288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:49.222673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] PATH_INFO: `/v3/users/b41ed92aeada435ea82e26acc6016578` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:49.226269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b41ed92aeada435ea82e26acc6016578)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:49.232708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:49.274553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 824/1648] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:49 2022] POST /identity/v3/users => generated 309 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:49.278902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:49.320049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b54159d-bd30-46e7-a96e-1153779bafbc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:49.341752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:49.343188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:49.343686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:49.344659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] PATH_INFO: `/v3/users/e8287678ddfb46d5a89f23a9c6f3dc00` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:49.349174 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8287678ddfb46d5a89f23a9c6f3dc00)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:49.352603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: WARNING py.warnings [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:25:49.352603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: warnings.warn(msg) Jul 07 10:25:49.352603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: Jul 07 10:25:49.358754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:49.413295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:49.441551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b41ed92aeada435ea82e26acc6016578'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.471058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b41ed92aeada435ea82e26acc6016578'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.471958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b41ed92aeada435ea82e26acc6016578'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.495577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b41ed92aeada435ea82e26acc6016578'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.524038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b41ed92aeada435ea82e26acc6016578'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.549336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 825/1649] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:25:48 2022] DELETE /identity/v3/users/b41ed92aeada435ea82e26acc6016578 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:49.549336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:49.551537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:49.568032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8287678ddfb46d5a89f23a9c6f3dc00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.595014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8287678ddfb46d5a89f23a9c6f3dc00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.596046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8287678ddfb46d5a89f23a9c6f3dc00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.638000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8287678ddfb46d5a89f23a9c6f3dc00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.663584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b249513f-3416-49bd-8f17-fe8cd160d478 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8287678ddfb46d5a89f23a9c6f3dc00'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:49.674249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 825/1650] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:25:49 2022] DELETE /identity/v3/users/e8287678ddfb46d5a89f23a9c6f3dc00 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:49.675871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac374e32-8137-4567-8a6a-9b74e0bcceac None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:49.676236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac374e32-8137-4567-8a6a-9b74e0bcceac None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:49.676592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac374e32-8137-4567-8a6a-9b74e0bcceac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:49.725800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ac374e32-8137-4567-8a6a-9b74e0bcceac None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:49.728792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ac374e32-8137-4567-8a6a-9b74e0bcceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:49.752515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61ec91a-b726-480f-b48e-97c902fcde47 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:49.879674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 826/1651] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:49.882072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:49.884877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:49.885353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:49.890502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:49.892667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:49.893375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:49.894533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:49.895388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:49.901383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:49.905318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:49.933937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 826/1652] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:49 2022] GET /identity/v3/roles => generated 1420 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:49.935025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:49.937210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:49.942044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:49.943273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:49.955371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 827/1653] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:49.956169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:49.958201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:49.975326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc7b193-c43a-41ac-b21d-eca9d02e2315 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:49.982242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:49.983167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:49.983449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:49.983725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] PATH_INFO: `/v3/projects/df3b29d6141b4a838c90695b6b98a6d0/users/0a4c247a460e4740b824fb7a417760c8/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:50.005581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3b29d6141b4a838c90695b6b98a6d0, user_id=0a4c247a460e4740b824fb7a417760c8, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:50.010070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:50.036665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 828/1654] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:49 2022] PUT /identity/v3/projects/df3b29d6141b4a838c90695b6b98a6d0/users/0a4c247a460e4740b824fb7a417760c8/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:50.037760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:50.041938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:50.058564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf90a80-d685-419a-9cbb-21df712a7292 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:50.083112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:50.124824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58cb582-2977-442b-a4ac-a448fa253e50 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:50.133839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:50.136009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:50.136306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:50.136583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:50.159602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:50.160719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:50.161504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:50.162153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:50.253378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:50.254835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:50.267812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 827/1655] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:50.269353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:50.278312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:50.279799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:50.280983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5495033e-c566-4297-9981-45023a1d8859 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:50.286766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:50.287518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:50.288155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:50.288748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:50.290050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:50.292309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:50.301109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 829/1656] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:50.350156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:50.350993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 828/1657] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:50.351082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ab996-a5c9-41f9-8502-56743964b4aa service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:50.351082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:50.351082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:50.351082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:50.351344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] PATH_INFO: `/v3/projects/df3b29d6141b4a838c90695b6b98a6d0/users/0a4c247a460e4740b824fb7a417760c8/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:50.351344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] RBAC: Authorizing `identity:create_grant(project_id=df3b29d6141b4a838c90695b6b98a6d0, user_id=0a4c247a460e4740b824fb7a417760c8, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:50.351344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:50.372011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 830/1658] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:50 2022] PUT /identity/v3/projects/df3b29d6141b4a838c90695b6b98a6d0/users/0a4c247a460e4740b824fb7a417760c8/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:25:51.848846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:51.850986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:51.872732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf26408-fc20-44e3-9525-4c7b2c3172e3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:52.034280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:52.035847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:52.036134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:52.036419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] PATH_INFO: `/v3/projects/7d14c724ddb346539e903734794ccf42` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:52.038454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d14c724ddb346539e903734794ccf42)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:52.039363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: WARNING py.warnings [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 07 10:25:52.039363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: warnings.warn(msg) Jul 07 10:25:52.039363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: Jul 07 10:25:52.043357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:52.051277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:52.054526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:52.076993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329939ed-6eb2-4b88-ae09-e06d3b43374d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:52.094618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:52.095769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:52.096339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:52.098630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] PATH_INFO: `/v3/projects/2c6a404439ce43e4b6210fcf354a5472` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:52.100481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c6a404439ce43e4b6210fcf354a5472)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:52.105627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:52.228802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d14c724ddb346539e903734794ccf42', 'request_id': 'req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.261037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c6a404439ce43e4b6210fcf354a5472', 'request_id': 'req-1cdad278-d679-4d4f-8f45-166f6606496e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.281334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d14c724ddb346539e903734794ccf42', 'request_id': 'req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.283781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:52.291691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c6a404439ce43e4b6210fcf354a5472', 'request_id': 'req-1cdad278-d679-4d4f-8f45-166f6606496e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.301558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d14c724ddb346539e903734794ccf42', 'request_id': 'req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.308434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c6a404439ce43e4b6210fcf354a5472', 'request_id': 'req-1cdad278-d679-4d4f-8f45-166f6606496e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.318527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:52.321683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] The token cache is being invalidate because project 7d14c724ddb346539e903734794ccf42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:25:52.322137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.324155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.343982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 829/1659] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:25:51 2022] DELETE /identity/v3/projects/7d14c724ddb346539e903734794ccf42 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:52.354206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] The token cache is being invalidate because project 2c6a404439ce43e4b6210fcf354a5472 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:25:52.354594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.356060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.357520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:52.361618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:52.363484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 831/1660] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:25:52 2022] DELETE /identity/v3/projects/2c6a404439ce43e4b6210fcf354a5472 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:52.364518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:52.366121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:52.547524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdad278-d679-4d4f-8f45-166f6606496e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:52.549502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ba5bb-0c54-40f3-9366-88b101c9e4ce admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:52.560902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:52.562283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:52.562578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:52.562955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] PATH_INFO: `/v3/users/a19558c8991f41aeb2b910cc27905f31` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:52.565299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:52.566214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:52.568481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a19558c8991f41aeb2b910cc27905f31)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:52.569879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:52.570212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] PATH_INFO: `/v3/users/973c27dbce9a40e59acd7753286f1cb6` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:52.572313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:52.573047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=973c27dbce9a40e59acd7753286f1cb6)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:52.576830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:52.718577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a19558c8991f41aeb2b910cc27905f31'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.732633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a19558c8991f41aeb2b910cc27905f31'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.735330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a19558c8991f41aeb2b910cc27905f31'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.739046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '973c27dbce9a40e59acd7753286f1cb6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.751365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '973c27dbce9a40e59acd7753286f1cb6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.752148 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '973c27dbce9a40e59acd7753286f1cb6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.766979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a19558c8991f41aeb2b910cc27905f31'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.769753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '973c27dbce9a40e59acd7753286f1cb6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.776183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-33194fa4-b03f-4f23-9316-65bc9f9ebee7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a19558c8991f41aeb2b910cc27905f31'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.787689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 832/1661] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:25:52 2022] DELETE /identity/v3/users/a19558c8991f41aeb2b910cc27905f31 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:52.789419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b45fb-db9f-4358-8035-bf0dccee4c4e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:52.789721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b45fb-db9f-4358-8035-bf0dccee4c4e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:52.790002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2b45fb-db9f-4358-8035-bf0dccee4c4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:52.821150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9e2b45fb-db9f-4358-8035-bf0dccee4c4e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:52.822917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e2b45fb-db9f-4358-8035-bf0dccee4c4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:52.837123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-8b363d0a-b9e6-4922-bdf3-fd3c7c45e286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '973c27dbce9a40e59acd7753286f1cb6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:52.857167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 830/1662] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:25:52 2022] DELETE /identity/v3/users/973c27dbce9a40e59acd7753286f1cb6 => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:52.861855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21b5e-56c0-403d-9e1e-f7b9b443704b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:52.862084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21b5e-56c0-403d-9e1e-f7b9b443704b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:52.862552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21b5e-56c0-403d-9e1e-f7b9b443704b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:52.917859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5eb21b5e-56c0-403d-9e1e-f7b9b443704b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:52.923791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5eb21b5e-56c0-403d-9e1e-f7b9b443704b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:52.929949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 833/1663] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:52.940832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb2d57-a410-40c4-8d46-561fd591904b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:52.941130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb2d57-a410-40c4-8d46-561fd591904b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:52.941415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb2d57-a410-40c4-8d46-561fd591904b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:52.965169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f2fb2d57-a410-40c4-8d46-561fd591904b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:52.966626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2fb2d57-a410-40c4-8d46-561fd591904b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:52.982558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:53.097623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 834/1664] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:53.101589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb2d57-a410-40c4-8d46-561fd591904b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:53.122567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb2d57-a410-40c4-8d46-561fd591904b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:53.132772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:53.133485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.133759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.134044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:53.134800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:53.135115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:53.139776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:53.162864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 831/1665] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:53.170678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 835/1666] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:25:53.174787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb21b5e-56c0-403d-9e1e-f7b9b443704b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:53.181962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:53.189609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb21b5e-56c0-403d-9e1e-f7b9b443704b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:53.203077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:53.204753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.205106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.206177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:53.213430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:53.216840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:53.236480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:53.238343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:53.310997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 832/1667] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:53 2022] POST /identity/v3/projects => generated 403 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:53.315512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:53.394752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4736c65f-b3c7-478d-b72a-44c0886590db admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:53.566673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db150ce4-2248-4c8c-95a3-06ebf7baf9fc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:53.600629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:53.601327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.601618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.601901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:53.612518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:53.613230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.613515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.613793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:53.614502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:53.616563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:53.641559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:53.643830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:53.665327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 833/1668] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:53 2022] POST /identity/v3/users => generated 302 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:25:53.671150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 836/1669] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:25:53.676864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:53.700898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099ca965-49f4-4f83-a0fd-0c3b3daf4c98 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:53.713253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:53.713960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.714247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.714527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:53.715830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:53.717477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:53.747757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 834/1670] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:25:53.752590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:53.766477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6678b1-29e3-444e-b541-109e54788088 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:53.779428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e06296-98f8-4515-b827-f125c105be7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:53.780181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e06296-98f8-4515-b827-f125c105be7a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.780474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e06296-98f8-4515-b827-f125c105be7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.780754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e06296-98f8-4515-b827-f125c105be7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:53.783460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e06296-98f8-4515-b827-f125c105be7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:53.788798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e06296-98f8-4515-b827-f125c105be7a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:53.817059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 837/1671] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:53 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:53.821471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:53.831900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701c663f-56fc-4588-940a-5478751be652 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:53.845900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:53.847006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.847140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.847382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] PATH_INFO: `/v3/projects/66585219154c4b15bc78b22930ad3e9d/users/bb889b16766e43f9a3ac76c48ae65c73/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:53.887630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=66585219154c4b15bc78b22930ad3e9d, user_id=bb889b16766e43f9a3ac76c48ae65c73, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:53.895584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:53.931012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 835/1672] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:53 2022] PUT /identity/v3/projects/66585219154c4b15bc78b22930ad3e9d/users/bb889b16766e43f9a3ac76c48ae65c73/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:53.945349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:53.945349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:53.945645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:54.023902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] MFA Rules not processed for user `bb889b16766e43f9a3ac76c48ae65c73`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:54.025223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:54.146051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:54.234940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 838/1673] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:53 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:54.239596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:54.431165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22084e4-7bee-4d6c-aee4-49ac24346f4d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:54.455537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:54.456880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:54.457172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:54.457457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:54.458166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:54.460177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:54.517019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:54.518791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:54.527756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 836/1674] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:54 2022] POST /identity/v3/projects => generated 405 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:54.531117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:54.575995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380bd5b-ee45-4196-aac3-78c2412b0785 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:54.625308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:54.708558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e92bd7d-ee45-4d15-a577-a24c2ba74e49 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:54.717242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:54.718218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:54.718512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:54.718792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:54.719621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:54.721697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:54.734193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:54.735110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:54.735399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:54.735687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] PATH_INFO: `/v3/users/25c62d6777c042baab53edee5556bdc3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:54.738773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=25c62d6777c042baab53edee5556bdc3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:54.741404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:54.772083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 837/1675] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:54 2022] POST /identity/v3/users => generated 309 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:54.773783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:54.776894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:54.798257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce3f21d-11ac-498f-a4ec-29ee4deae481 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:54.815077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:54.815792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:54.816284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:54.816463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] PATH_INFO: `/v3/projects/5751156d7da241b486bad5d9434133a5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:54.817723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5751156d7da241b486bad5d9434133a5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:54.820168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:54.851374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25c62d6777c042baab53edee5556bdc3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.875449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25c62d6777c042baab53edee5556bdc3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.876123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25c62d6777c042baab53edee5556bdc3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.888028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5751156d7da241b486bad5d9434133a5', 'request_id': 'req-ab2bf586-4180-4627-a1b5-9c899643f12c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.899060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25c62d6777c042baab53edee5556bdc3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.902389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5751156d7da241b486bad5d9434133a5', 'request_id': 'req-ab2bf586-4180-4627-a1b5-9c899643f12c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.909773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25c62d6777c042baab53edee5556bdc3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.921459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5751156d7da241b486bad5d9434133a5', 'request_id': 'req-ab2bf586-4180-4627-a1b5-9c899643f12c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.935622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 839/1676] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:25:54 2022] DELETE /identity/v3/users/25c62d6777c042baab53edee5556bdc3 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:54.936825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:54.938552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:54.954386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] The token cache is being invalidate because project 5751156d7da241b486bad5d9434133a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:25:54.954864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.963709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:54.967663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 838/1677] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:25:54 2022] DELETE /identity/v3/projects/5751156d7da241b486bad5d9434133a5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:54.972512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:54.976228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.098459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:55.103010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b3125e-b5ab-49e7-9d1e-a14190aeae2e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:55.118248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56086e7-4089-4261-98da-70e6ff87007c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:55.119252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56086e7-4089-4261-98da-70e6ff87007c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.119549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56086e7-4089-4261-98da-70e6ff87007c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.119867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56086e7-4089-4261-98da-70e6ff87007c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.120629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56086e7-4089-4261-98da-70e6ff87007c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:55.122844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56086e7-4089-4261-98da-70e6ff87007c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:55.142522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 840/1678] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:54 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:55.144036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65598-a714-44df-b16a-a24353cb09b5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.144339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65598-a714-44df-b16a-a24353cb09b5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.144626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae65598-a714-44df-b16a-a24353cb09b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.159324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2bf586-4180-4627-a1b5-9c899643f12c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:55.166693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:55.167697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.168699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.169087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] PATH_INFO: `/v3/projects/c8f89082d97f4967910ab5dccc941265` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.170482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8f89082d97f4967910ab5dccc941265)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:55.172964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:55.199866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-eae65598-a714-44df-b16a-a24353cb09b5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:55.201137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eae65598-a714-44df-b16a-a24353cb09b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.229428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 841/1679] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:55.229428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.229428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.229428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.276267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:55.278799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:55.281171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.303362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8f89082d97f4967910ab5dccc941265', 'request_id': 'req-6d9aa35f-c575-4e09-8f4e-5f46e492c794'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:55.323901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8f89082d97f4967910ab5dccc941265', 'request_id': 'req-6d9aa35f-c575-4e09-8f4e-5f46e492c794'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:55.349392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8f89082d97f4967910ab5dccc941265', 'request_id': 'req-6d9aa35f-c575-4e09-8f4e-5f46e492c794'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:55.370756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] The token cache is being invalidate because project c8f89082d97f4967910ab5dccc941265 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:25:55.373471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:55.374594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:55.378569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 839/1680] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:25:54 2022] DELETE /identity/v3/projects/c8f89082d97f4967910ab5dccc941265 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:55.383805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:55.390778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.473217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 842/1681] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:55.474295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:55.481913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.571150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9aa35f-c575-4e09-8f4e-5f46e492c794 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:55.586052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe43f97-c95a-4bcb-add9-698d4432efc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:55.586986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43f97-c95a-4bcb-add9-698d4432efc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.587273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43f97-c95a-4bcb-add9-698d4432efc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.587705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe43f97-c95a-4bcb-add9-698d4432efc7 admin admin] PATH_INFO: `/v3/projects/708099e248894d6b89b7eca0e4cdc436/users/a8a162a72eae4ee4a831d4a53bec728b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.619415 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43f97-c95a-4bcb-add9-698d4432efc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=708099e248894d6b89b7eca0e4cdc436, user_id=a8a162a72eae4ee4a831d4a53bec728b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:55.622751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe43f97-c95a-4bcb-add9-698d4432efc7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:55.636941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee9ff1-45b5-4f31-b00c-aa5e343ca5c7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:55.644472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dadf0bb-ac62-4ad0-a6a8-dd112a277387 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:55.646057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadf0bb-ac62-4ad0-a6a8-dd112a277387 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.646515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadf0bb-ac62-4ad0-a6a8-dd112a277387 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.647002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dadf0bb-ac62-4ad0-a6a8-dd112a277387 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.655221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4dadf0bb-ac62-4ad0-a6a8-dd112a277387 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.657898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 840/1682] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:55 2022] PUT /identity/v3/projects/708099e248894d6b89b7eca0e4cdc436/users/a8a162a72eae4ee4a831d4a53bec728b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:55.659029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc7d2-3e7d-4a5e-a0f4-98e0ab3373d8 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.659324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc7d2-3e7d-4a5e-a0f4-98e0ab3373d8 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.659614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc7d2-3e7d-4a5e-a0f4-98e0ab3373d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.672264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadf0bb-ac62-4ad0-a6a8-dd112a277387 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:55.674720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dadf0bb-ac62-4ad0-a6a8-dd112a277387 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:55.690288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 843/1683] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:55.691877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0e1d9-2eae-4e92-a6a8-0f0111e60396 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.692306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0e1d9-2eae-4e92-a6a8-0f0111e60396 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.692746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb0e1d9-2eae-4e92-a6a8-0f0111e60396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.717308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-df7fc7d2-3e7d-4a5e-a0f4-98e0ab3373d8 None None] MFA Rules not processed for user `0a4c247a460e4740b824fb7a417760c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:55.718561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-df7fc7d2-3e7d-4a5e-a0f4-98e0ab3373d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.719614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0bb0e1d9-2eae-4e92-a6a8-0f0111e60396 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:55.724045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0bb0e1d9-2eae-4e92-a6a8-0f0111e60396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.869802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 841/1684] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:55.875508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd28dd-66c8-42ec-8032-6c67c28bee53 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.875508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd28dd-66c8-42ec-8032-6c67c28bee53 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.875508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd28dd-66c8-42ec-8032-6c67c28bee53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.920368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 844/1685] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:55.921925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0e1d9-2eae-4e92-a6a8-0f0111e60396 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:55.933449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cbcd28dd-66c8-42ec-8032-6c67c28bee53 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:55.934714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cbcd28dd-66c8-42ec-8032-6c67c28bee53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:55.943710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb0e1d9-2eae-4e92-a6a8-0f0111e60396 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:55.951819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 842/1686] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:55.953223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe41a5e-2082-4d8d-b7bb-862290f2187c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.953516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe41a5e-2082-4d8d-b7bb-862290f2187c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.953795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe41a5e-2082-4d8d-b7bb-862290f2187c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.957996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:55.958909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:55.959341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:55.959819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:55.962241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:55.964068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:55.995871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 845/1687] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:55.997513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:55.997807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bbe41a5e-2082-4d8d-b7bb-862290f2187c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:55.999366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bbe41a5e-2082-4d8d-b7bb-862290f2187c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:56.023510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 843/1688] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:56.024365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe41a5e-2082-4d8d-b7bb-862290f2187c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.025286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01b30ce-5f7f-40a4-9369-b408d8e6ec8b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.036091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.038918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.039381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.040701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.040949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe41a5e-2082-4d8d-b7bb-862290f2187c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.043295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:56.043983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.046017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.054473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.055564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.055885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.056167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] PATH_INFO: `/v3/projects/708099e248894d6b89b7eca0e4cdc436/users/a8a162a72eae4ee4a831d4a53bec728b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.060939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] RBAC: Authorizing `identity:create_grant(project_id=708099e248894d6b89b7eca0e4cdc436, user_id=a8a162a72eae4ee4a831d4a53bec728b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.063648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.088633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 846/1689] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:56.089422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.116261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea074e86-4dbe-430b-b0f7-eb4f521e89ef admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.116454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 844/1690] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:56 2022] PUT /identity/v3/projects/708099e248894d6b89b7eca0e4cdc436/users/a8a162a72eae4ee4a831d4a53bec728b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:56.117351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.139275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:56.141023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:56.269580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257cf9cb-00fa-409c-bc98-4e10811f886c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.270451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257cf9cb-00fa-409c-bc98-4e10811f886c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.270847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257cf9cb-00fa-409c-bc98-4e10811f886c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.271244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257cf9cb-00fa-409c-bc98-4e10811f886c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.272136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257cf9cb-00fa-409c-bc98-4e10811f886c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:25:56.272708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257cf9cb-00fa-409c-bc98-4e10811f886c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.274455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257cf9cb-00fa-409c-bc98-4e10811f886c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.276942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e5b767-e80f-4f7f-a51d-f7e1e79fe915 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.292137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 847/1691] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:25:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:25:56.292137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcea46e-05bc-4ea5-bf71-3b6089a77203 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.292137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcea46e-05bc-4ea5-bf71-3b6089a77203 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.293624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcea46e-05bc-4ea5-bf71-3b6089a77203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.294620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.295440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.295731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.296034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.296733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.298689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.333283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1bcea46e-05bc-4ea5-bf71-3b6089a77203 None None] MFA Rules not processed for user `a8a162a72eae4ee4a831d4a53bec728b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:56.334579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1bcea46e-05bc-4ea5-bf71-3b6089a77203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:56.355093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 845/1692] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:56 2022] POST /identity/v3/projects => generated 365 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:56.355093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.441847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 848/1693] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:56 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:56.444266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcea46e-05bc-4ea5-bf71-3b6089a77203 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.466909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcea46e-05bc-4ea5-bf71-3b6089a77203 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.472577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e05e12-c853-4a84-9a3a-bf269b379b4e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.481647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.483587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.484450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.484742 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.485489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.489866 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.490505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.490797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.491643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.493297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.494425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.497922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.546546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 849/1694] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:56 2022] POST /identity/v3/users => generated 283 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:56.550319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.552237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:56.575243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 846/1695] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:56 2022] POST /identity/v3/projects => generated 387 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:56.575243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.687418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c7184f-5f64-4e96-ba8d-3ce2aea95df1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.693644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.694767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.699150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.699544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.700363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.702729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.703017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e122accc-6812-4414-87d3-a4a43eeba3c2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.724529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 847/1696] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:56.724529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.744408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:56.746689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab48c52d-a40b-4380-a2f8-12f9573ee770 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.761974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.762956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.763563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.764049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.765550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.767681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.783713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.784499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.784789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.785067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.799850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:56.824900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 848/1697] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:56 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:56.824900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.862890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d44c12c-ae2f-414c-b06d-aff898de5a49 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.889971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.890858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.890993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.890993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.891276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.892063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.893790 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.895549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.919164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 850/1698] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:56 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:56.924009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.939985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 849/1699] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:56.942587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.943506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945afc20-99b3-45e9-9115-36bf2e81ef10 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.950124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.952051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.952398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.952685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:56.953438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:56.955383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:56.972285 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b84017-d944-4de7-abd4-a9d56866c64f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:56.975796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 851/1700] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:56.979206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:56.989706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f266dc2-f517-4ba5-95e8-6236f813ac58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:56.991237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266dc2-f517-4ba5-95e8-6236f813ac58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:56.991688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266dc2-f517-4ba5-95e8-6236f813ac58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:56.992218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f266dc2-f517-4ba5-95e8-6236f813ac58 admin admin] PATH_INFO: `/v3/projects/48a8062587cb41da8711b1cc21e401ec/users/a86096a61a6a4b339403128e5dc16ba9/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.017880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9fbe81-7077-48ca-8002-b31af355ae43 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:57.030560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:57.031260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.042152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266dc2-f517-4ba5-95e8-6236f813ac58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48a8062587cb41da8711b1cc21e401ec, user_id=a86096a61a6a4b339403128e5dc16ba9, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:57.047045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f266dc2-f517-4ba5-95e8-6236f813ac58 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:57.048387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.050421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.052485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:57.062795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:57.077976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 850/1701] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:56 2022] PUT /identity/v3/projects/48a8062587cb41da8711b1cc21e401ec/users/a86096a61a6a4b339403128e5dc16ba9/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:25:57.088102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296c723-e988-4545-8dc5-389b39b84e75 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.088775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296c723-e988-4545-8dc5-389b39b84e75 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.089250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296c723-e988-4545-8dc5-389b39b84e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.110017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 852/1702] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:57.114519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:57.133256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36067300-4888-4018-81c4-8babc7bbbe0e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:57.150809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c296c723-e988-4545-8dc5-389b39b84e75 None None] MFA Rules not processed for user `a86096a61a6a4b339403128e5dc16ba9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:57.152214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c296c723-e988-4545-8dc5-389b39b84e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:57.261529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:57.263527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.263846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.264125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] PATH_INFO: `/v3/projects/88295873db5d48bbb3f65410cda48aa8/users/674800b16791439884c2fa8588eae873/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.274515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 851/1703] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:25:57.276401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296c723-e988-4545-8dc5-389b39b84e75 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:57.283465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=88295873db5d48bbb3f65410cda48aa8, user_id=674800b16791439884c2fa8588eae873, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:57.286718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:57.292238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296c723-e988-4545-8dc5-389b39b84e75 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:57.299495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:57.300729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.301012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.301286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] PATH_INFO: `/v3/users/bb889b16766e43f9a3ac76c48ae65c73` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.302950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb889b16766e43f9a3ac76c48ae65c73)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:57.305258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:57.319757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 853/1704] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:57 2022] PUT /identity/v3/projects/88295873db5d48bbb3f65410cda48aa8/users/674800b16791439884c2fa8588eae873/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:57.322399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:57.390714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb889b16766e43f9a3ac76c48ae65c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:57.403500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb889b16766e43f9a3ac76c48ae65c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:57.404602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb889b16766e43f9a3ac76c48ae65c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:57.417174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb889b16766e43f9a3ac76c48ae65c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:57.420609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:57.444763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-89fc7cbe-58f5-479f-ac35-9ee177c3f003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb889b16766e43f9a3ac76c48ae65c73'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:57.457390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 852/1705] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:25:57 2022] DELETE /identity/v3/users/bb889b16766e43f9a3ac76c48ae65c73 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:57.459727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0846c-4505-427f-9bd5-7b27a868520c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.460283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0846c-4505-427f-9bd5-7b27a868520c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.460447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c0846c-4505-427f-9bd5-7b27a868520c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.475319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36c70ae-75c0-4ea7-9aee-56d799148044 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:57.502494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a8c0846c-4505-427f-9bd5-7b27a868520c None None] MFA Rules not processed for user `674800b16791439884c2fa8588eae873`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:57.503909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a8c0846c-4505-427f-9bd5-7b27a868520c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:57.523915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:57.629213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:57.630115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.630404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.630684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.631387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:57.631488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 853/1706] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:57.636616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:57.637604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.638029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.638318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.672812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:57.674070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:57.689374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 854/1707] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:57.690416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:57.691931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:57.709129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 854/1708] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:57 2022] POST /identity/v3/projects => generated 363 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:57.710097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:57.712584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:57.721137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610298e-41d2-4f03-b76b-58c714e6daf6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:57.859999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb197e88-e00a-47c5-89ed-9d0591640479 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:57.872192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:57.873714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.874013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.875372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.880650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:57.881901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:57.882194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:57.883013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:57.883727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:57.889287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:57.897925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:57.901940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:57.944923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 855/1709] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:57.948360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:57.960900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:57.974849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 855/1710] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:25:57 2022] POST /identity/v3/projects => generated 385 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:57.978740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:57.981130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:57.993058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deefe90b-7a49-4431-8078-9a1902218313 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:58.116145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:58.116832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.117118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.117398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.118364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:58.120971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:58.137161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5563c100-eee5-4383-80dc-1eab397cb8a7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:58.142883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:58.143784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.144074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.144351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] PATH_INFO: `/v3/projects/bf455075addb494f89fc180a2c0fc19f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.147962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf455075addb494f89fc180a2c0fc19f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:58.150482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:58.155830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:58.202687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 856/1711] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:57 2022] POST /identity/v3/users => generated 288 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:58.203461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:58.218832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15725dd8-3d25-4727-a77d-3909b1ad3553 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:58.228888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1a5d3-fd62-4676-a34d-90fbac0f756e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:58.229616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1a5d3-fd62-4676-a34d-90fbac0f756e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.230201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1a5d3-fd62-4676-a34d-90fbac0f756e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.231127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1a5d3-fd62-4676-a34d-90fbac0f756e admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.231975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1a5d3-fd62-4676-a34d-90fbac0f756e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:58.237616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d1a5d3-fd62-4676-a34d-90fbac0f756e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:58.246165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:58.285310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf455075addb494f89fc180a2c0fc19f', 'request_id': 'req-e95ca835-45e3-4c9f-bd0d-623695767eb3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.287130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 857/1712] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:25:58 2022] POST /identity/v3/users => generated 299 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:25:58.290352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.290880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.291357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.303534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf455075addb494f89fc180a2c0fc19f', 'request_id': 'req-e95ca835-45e3-4c9f-bd0d-623695767eb3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.323627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf455075addb494f89fc180a2c0fc19f', 'request_id': 'req-e95ca835-45e3-4c9f-bd0d-623695767eb3'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.342018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] The token cache is being invalidate because project bf455075addb494f89fc180a2c0fc19f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:25:58.342336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.345812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.346144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] MFA Rules not processed for user `247787cd745444f38953a64780d6ac9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:58.347796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:58.351859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 856/1713] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:25:57 2022] DELETE /identity/v3/projects/bf455075addb494f89fc180a2c0fc19f => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:58.352830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:58.356891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:58.505035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95ca835-45e3-4c9f-bd0d-623695767eb3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:58.536060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:58.536770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.537060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.537482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.538547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:58.541429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:58.560604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 857/1714] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:58.566670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:58.575764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:58.587469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 858/1715] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:58.588832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:58.593061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:58.601599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c610e04-c483-4f2b-b494-38d02666639f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:58.617358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:58.618057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.618342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.618623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.619370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:58.619774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a39f5-76c3-45bd-9533-635d34e09435 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:58.621065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:58.626187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:58.630049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.630372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.630773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] PATH_INFO: `/v3/users/9e420abe8b864523bdf21fbddde8677d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.643368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e420abe8b864523bdf21fbddde8677d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:58.645677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:58.646418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 858/1716] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:58.650200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:58.671604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524f1941-33a1-46ad-af44-58c6d448f7f2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:58.686149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:58.686702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:58.686954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:58.687917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] PATH_INFO: `/v3/projects/4e8c7fc738f54d80bd58d5d32f2fe7d7/users/839d13367e1649babf76876157e8a364/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:58.729440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8c7fc738f54d80bd58d5d32f2fe7d7, user_id=839d13367e1649babf76876157e8a364, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:58.733718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:58.740274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e420abe8b864523bdf21fbddde8677d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.755370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e420abe8b864523bdf21fbddde8677d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.756170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e420abe8b864523bdf21fbddde8677d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.768294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 859/1717] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:58 2022] PUT /identity/v3/projects/4e8c7fc738f54d80bd58d5d32f2fe7d7/users/839d13367e1649babf76876157e8a364/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:58.769235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:58.776724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:58.806163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e420abe8b864523bdf21fbddde8677d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.823560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e420abe8b864523bdf21fbddde8677d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:25:58.837230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 859/1718] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:25:58 2022] DELETE /identity/v3/users/9e420abe8b864523bdf21fbddde8677d => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:58.839714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:58.846314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:58.970455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:58.982795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac175d-372b-4c5b-811a-731e2ea6b501 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:59.006125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edde603-adb4-4b0a-9d0c-a9f423db7187 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:59.013815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:59.019345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:59.020331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.020638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.021031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] PATH_INFO: `/v3/projects/2130f287560041b5bcd75970b2adb316/users/c1a48833654c426282fe7a4fa6ea305b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.060520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2130f287560041b5bcd75970b2adb316, user_id=c1a48833654c426282fe7a4fa6ea305b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:59.062651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:59.090692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 860/1719] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:58 2022] PUT /identity/v3/projects/2130f287560041b5bcd75970b2adb316/users/c1a48833654c426282fe7a4fa6ea305b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:59.091484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:59.096834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:59.112545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803bc4a7-17b3-4368-9685-d3dd1ebfce92 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:59.113269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bc4a7-17b3-4368-9685-d3dd1ebfce92 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.113560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bc4a7-17b3-4368-9685-d3dd1ebfce92 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.113839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803bc4a7-17b3-4368-9685-d3dd1ebfce92 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.117203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-803bc4a7-17b3-4368-9685-d3dd1ebfce92 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:59.238594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d55c067-112b-438b-9b4a-2f680c05abd6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:59.244279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bc4a7-17b3-4368-9685-d3dd1ebfce92 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:59.246729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803bc4a7-17b3-4368-9685-d3dd1ebfce92 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:59.254719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:59.255547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.255947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.256303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.257338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:59.259060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:59.287908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 860/1720] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:25:59.296388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f210-ed3d-4d97-bbc9-5df16876e544 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.297417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f210-ed3d-4d97-bbc9-5df16876e544 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.297653 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a415f210-ed3d-4d97-bbc9-5df16876e544 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.303007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 861/1721] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:25:59 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:25:59.307697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415f210-ed3d-4d97-bbc9-5df16876e544 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:59.316109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 861/1722] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:59.324626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:59.370195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a60eac-7b18-4c8a-8986-d4bfd8d6e463 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:59.372651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a415f210-ed3d-4d97-bbc9-5df16876e544 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:59.388188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65db4e3-bc42-40cf-922b-286c7e173d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:59.389082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65db4e3-bc42-40cf-922b-286c7e173d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.393096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65db4e3-bc42-40cf-922b-286c7e173d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.393521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65db4e3-bc42-40cf-922b-286c7e173d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.394268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:59.395283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65db4e3-bc42-40cf-922b-286c7e173d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:59.398595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.398735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65db4e3-bc42-40cf-922b-286c7e173d3c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:59.400239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.402675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] PATH_INFO: `/v3/projects/4e8c7fc738f54d80bd58d5d32f2fe7d7/users/839d13367e1649babf76876157e8a364/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.408121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e8c7fc738f54d80bd58d5d32f2fe7d7, user_id=839d13367e1649babf76876157e8a364, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:59.417771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:59.430149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 862/1723] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:25:59 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:25:59.437604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e4536-61a1-420d-ae1a-35c8d76d890d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.439718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e4536-61a1-420d-ae1a-35c8d76d890d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.440236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01e4536-61a1-420d-ae1a-35c8d76d890d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.469931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 862/1724] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:59 2022] PUT /identity/v3/projects/4e8c7fc738f54d80bd58d5d32f2fe7d7/users/839d13367e1649babf76876157e8a364/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:59.469931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:59.502503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a01e4536-61a1-420d-ae1a-35c8d76d890d None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:59.505559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a01e4536-61a1-420d-ae1a-35c8d76d890d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:59.655014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8888d37-d01a-421b-8db4-18a95686e73e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:25:59.675046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58749136-f560-402e-bf3b-be1dd04050a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:25:59.676603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58749136-f560-402e-bf3b-be1dd04050a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.677282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58749136-f560-402e-bf3b-be1dd04050a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.677763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58749136-f560-402e-bf3b-be1dd04050a6 admin admin] PATH_INFO: `/v3/projects/2130f287560041b5bcd75970b2adb316/users/c1a48833654c426282fe7a4fa6ea305b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.681410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58749136-f560-402e-bf3b-be1dd04050a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2130f287560041b5bcd75970b2adb316, user_id=c1a48833654c426282fe7a4fa6ea305b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:25:59.688129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58749136-f560-402e-bf3b-be1dd04050a6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:25:59.695145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 863/1725] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:59.696575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943f8d-4dc8-4d3d-a696-749bd3f274fa None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.696869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943f8d-4dc8-4d3d-a696-749bd3f274fa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.697150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52943f8d-4dc8-4d3d-a696-749bd3f274fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.742169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 863/1726] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:25:59 2022] PUT /identity/v3/projects/2130f287560041b5bcd75970b2adb316/users/c1a48833654c426282fe7a4fa6ea305b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:25:59.744222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba962e-4fa0-42e7-a0d5-215469d06094 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.744744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba962e-4fa0-42e7-a0d5-215469d06094 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.745224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ba962e-4fa0-42e7-a0d5-215469d06094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.761943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-52943f8d-4dc8-4d3d-a696-749bd3f274fa None None] MFA Rules not processed for user `839d13367e1649babf76876157e8a364`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:59.763222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-52943f8d-4dc8-4d3d-a696-749bd3f274fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:59.797324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-25ba962e-4fa0-42e7-a0d5-215469d06094 None None] MFA Rules not processed for user `c1a48833654c426282fe7a4fa6ea305b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:59.799213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-25ba962e-4fa0-42e7-a0d5-215469d06094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:59.836476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:59.888329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:25:59.940091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 864/1727] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:59.943954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349acc-f2c6-4811-bd1a-8f791881e4d6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.944838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349acc-f2c6-4811-bd1a-8f791881e4d6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.945129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc349acc-f2c6-4811-bd1a-8f791881e4d6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.946807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 865/1728] 10.222.0.16 () {54 vars in 960 bytes} [Thu Jul 7 10:25:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:25:59.951204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349acc-f2c6-4811-bd1a-8f791881e4d6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:25:59.952905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 864/1729] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:25:59.954576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:25:59.955907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:25:59.956460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:25:59.961310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fc349acc-f2c6-4811-bd1a-8f791881e4d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:25:59.995938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:25:59.998192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:00.116601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc349acc-f2c6-4811-bd1a-8f791881e4d6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:00.124804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f42af1a-f013-4df4-94cd-ed28af50bd2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:00.126807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42af1a-f013-4df4-94cd-ed28af50bd2b service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.127265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42af1a-f013-4df4-94cd-ed28af50bd2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.128397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f42af1a-f013-4df4-94cd-ed28af50bd2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.133099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 865/1730] 10.222.0.16 () {58 vars in 1057 bytes} [Thu Jul 7 10:25:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:00.134094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:00.136404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:00.154553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42af1a-f013-4df4-94cd-ed28af50bd2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:00.154660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd099e7-feae-4013-ac55-e9ec6150f4f5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:00.156028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f42af1a-f013-4df4-94cd-ed28af50bd2b service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:00.161588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:00.162416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.162703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.162981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.180803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 866/1731] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:25:59 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:00.182303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1c08b-2f4c-4bd3-bdee-07d1c3965cd1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.182598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1c08b-2f4c-4bd3-bdee-07d1c3965cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.182876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b1c08b-2f4c-4bd3-bdee-07d1c3965cd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.204727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:00.208853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:00.240890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 866/1732] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:00.241801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:00.245062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c1b1c08b-2f4c-4bd3-bdee-07d1c3965cd1 None None] MFA Rules not processed for user `92b758b22f054dd991e8af99354a134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:00.248582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1b1c08b-2f4c-4bd3-bdee-07d1c3965cd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:00.287236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dc079-ba58-4cdf-a249-cf9a55a36bc8 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:00.329636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:00.330374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.330642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.330921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.359140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:00.362709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:00.389202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 867/1733] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:00.389878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:00.421236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 867/1734] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:00.431717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1c08b-2f4c-4bd3-bdee-07d1c3965cd1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:00.452915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b1c08b-2f4c-4bd3-bdee-07d1c3965cd1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:00.464023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6caa3de-f6f8-48d9-9e03-38aaa95e69e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:00.464696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caa3de-f6f8-48d9-9e03-38aaa95e69e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.465017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caa3de-f6f8-48d9-9e03-38aaa95e69e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.465296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6caa3de-f6f8-48d9-9e03-38aaa95e69e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.468916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6caa3de-f6f8-48d9-9e03-38aaa95e69e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:00.581368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dedc4c-453c-4231-945d-8519f9715c49 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:00.586866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:00.587721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.588050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.588545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] PATH_INFO: `/v3/users/674800b16791439884c2fa8588eae873` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.591283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=674800b16791439884c2fa8588eae873)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:00.596034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:00.645980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:00.653560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caa3de-f6f8-48d9-9e03-38aaa95e69e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:00.659428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6caa3de-f6f8-48d9-9e03-38aaa95e69e2 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:00.681620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 868/1735] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:00 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:00.682874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037198fc-c1c8-4969-95a1-347b3d9f5890 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.683163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037198fc-c1c8-4969-95a1-347b3d9f5890 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.684338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037198fc-c1c8-4969-95a1-347b3d9f5890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.692442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:00.703727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '674800b16791439884c2fa8588eae873'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:00.717296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '674800b16791439884c2fa8588eae873'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:00.717953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '674800b16791439884c2fa8588eae873'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:00.745931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '674800b16791439884c2fa8588eae873'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:00.763515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '674800b16791439884c2fa8588eae873'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:00.783162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 868/1736] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:00 2022] DELETE /identity/v3/users/674800b16791439884c2fa8588eae873 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:00.785078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:00.791189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:00.800094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-037198fc-c1c8-4969-95a1-347b3d9f5890 None None] MFA Rules not processed for user `92b758b22f054dd991e8af99354a134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:00.805706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-037198fc-c1c8-4969-95a1-347b3d9f5890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:00.970156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca29f38-2fef-44aa-af07-7ec2e94462b8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:00.984798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:00.985752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:00.986043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:00.986411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] PATH_INFO: `/v3/projects/66585219154c4b15bc78b22930ad3e9d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:00.989176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] RBAC: Authorizing `identity:delete_project(project_id=66585219154c4b15bc78b22930ad3e9d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:00.995871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:01.028018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 869/1737] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:00 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:01.035722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:01.036041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:01.036320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:01.108979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:01.110240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:01.154217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66585219154c4b15bc78b22930ad3e9d', 'request_id': 'req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.185853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66585219154c4b15bc78b22930ad3e9d', 'request_id': 'req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.211388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66585219154c4b15bc78b22930ad3e9d', 'request_id': 'req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.271338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] The token cache is being invalidate because project 66585219154c4b15bc78b22930ad3e9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:01.271676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.280037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.286981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 869/1738] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:00 2022] DELETE /identity/v3/projects/66585219154c4b15bc78b22930ad3e9d => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:01.287910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:01.301693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:01.407260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 870/1739] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:01.408282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:01.411339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:01.432016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf8d57b-3dc1-4755-a782-f2ad391db54c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:01.445189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:01.446207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:01.446489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:01.446763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] PATH_INFO: `/v3/users/a86096a61a6a4b339403128e5dc16ba9` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:01.451791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a86096a61a6a4b339403128e5dc16ba9)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:01.458641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:01.512821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:01.558045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a86096a61a6a4b339403128e5dc16ba9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.563118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d895bb4c-0bfc-46e7-aff5-ab0aed4e493f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:01.575196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07cf67a-c082-4d0b-984e-65ce1c96e6e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:01.576366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf67a-c082-4d0b-984e-65ce1c96e6e3 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:01.577008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf67a-c082-4d0b-984e-65ce1c96e6e3 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:01.577133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a86096a61a6a4b339403128e5dc16ba9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.577747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07cf67a-c082-4d0b-984e-65ce1c96e6e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:01.577935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a86096a61a6a4b339403128e5dc16ba9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.581044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e07cf67a-c082-4d0b-984e-65ce1c96e6e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:01.588863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a86096a61a6a4b339403128e5dc16ba9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.606479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a86096a61a6a4b339403128e5dc16ba9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:01.617273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 871/1740] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:01 2022] DELETE /identity/v3/users/a86096a61a6a4b339403128e5dc16ba9 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:01.618402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:01.619974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:01.650896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:01.790776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cf67a-c082-4d0b-984e-65ce1c96e6e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:01.795542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07cf67a-c082-4d0b-984e-65ce1c96e6e3 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:01.798490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3837bda-e0c5-4b9f-9d41-6569812233f5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:01.821749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 870/1741] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:01.823904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:01.824243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:01.827971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:01.833464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:01.834577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:01.835007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:01.836048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] PATH_INFO: `/v3/users/a8a162a72eae4ee4a831d4a53bec728b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:01.839036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8a162a72eae4ee4a831d4a53bec728b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:01.849285 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:01.963933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:01.968120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:01.995129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 871/1742] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:01.996162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:02.001064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8a162a72eae4ee4a831d4a53bec728b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.001419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:02.014843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8a162a72eae4ee4a831d4a53bec728b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.015537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8a162a72eae4ee4a831d4a53bec728b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.023540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8a162a72eae4ee4a831d4a53bec728b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.058052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8a162a72eae4ee4a831d4a53bec728b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.072754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 872/1743] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:01 2022] DELETE /identity/v3/users/a8a162a72eae4ee4a831d4a53bec728b => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:02.077801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:02.079687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:02.191617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7b3db-515d-4def-9d33-2b1964446fda None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:02.344620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:02.345371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:02.351778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:02.352230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:02.365833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:02.516536 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:02.557149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:02.564615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:02.580003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b04032-3a9e-4edd-a1f3-9befb72fc05f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:02.594409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 872/1744] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 599 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:02.595296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:02.595611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:02.596497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:02.596903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:02.597325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] PATH_INFO: `/v3/projects/1a6c8ef820104ed09e026d7c92ec5a26` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:02.597635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:02.601313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a6c8ef820104ed09e026d7c92ec5a26)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:02.604347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:02.615666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86861f8-6cf3-4a8f-a395-41faa232e939 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:02.624024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:02.624720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:02.625045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:02.625329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:02.634856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:02.654225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:02.656284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:02.672252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 873/1745] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:02.697175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:02.724624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a6c8ef820104ed09e026d7c92ec5a26', 'request_id': 'req-5899b49c-d8d4-4a51-9341-b2941955773f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.734671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a6c8ef820104ed09e026d7c92ec5a26', 'request_id': 'req-5899b49c-d8d4-4a51-9341-b2941955773f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.736233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:02.738470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:02.762633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279cf967-94c9-4d5a-b11e-acbf01c38f18 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:02.764853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a6c8ef820104ed09e026d7c92ec5a26', 'request_id': 'req-5899b49c-d8d4-4a51-9341-b2941955773f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.792485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] The token cache is being invalidate because project 1a6c8ef820104ed09e026d7c92ec5a26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:02.792809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.797287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5899b49c-d8d4-4a51-9341-b2941955773f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:02.807255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 873/1746] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:02 2022] DELETE /identity/v3/projects/1a6c8ef820104ed09e026d7c92ec5a26 => generated 0 bytes in 725 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:02.811565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d631254-37ee-4f73-b936-a432287840cf None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:02.812354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d631254-37ee-4f73-b936-a432287840cf None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:02.814503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d631254-37ee-4f73-b936-a432287840cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:02.868520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5d631254-37ee-4f73-b936-a432287840cf None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:02.873014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5d631254-37ee-4f73-b936-a432287840cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:03.000471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:03.003634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.003971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.004287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] PATH_INFO: `/v3/users/32d8a8e3820348d9885b8447104fb72c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.007086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d8a8e3820348d9885b8447104fb72c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:03.009100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:03.098015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 874/1747] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:03.100422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedee659-71bc-4cb6-bc30-03a39524bf5b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.101540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedee659-71bc-4cb6-bc30-03a39524bf5b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.102030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedee659-71bc-4cb6-bc30-03a39524bf5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.134054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d8a8e3820348d9885b8447104fb72c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:03.147703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-dedee659-71bc-4cb6-bc30-03a39524bf5b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:03.149370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d8a8e3820348d9885b8447104fb72c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:03.150250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d8a8e3820348d9885b8447104fb72c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:03.154414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dedee659-71bc-4cb6-bc30-03a39524bf5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:03.176187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 875/1748] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:03.180281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d8a8e3820348d9885b8447104fb72c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:03.183716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedee659-71bc-4cb6-bc30-03a39524bf5b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:03.205943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d8a8e3820348d9885b8447104fb72c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:03.223388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedee659-71bc-4cb6-bc30-03a39524bf5b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:03.236290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 874/1749] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:02 2022] DELETE /identity/v3/users/32d8a8e3820348d9885b8447104fb72c => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:03.241688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:03.242899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.243294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.243945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.244938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:03.245558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:03.248292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:03.313449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 876/1750] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:26:03.315815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:03.432511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:03.493158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c8ae0-1d7d-4600-b7b5-0ea574ddaf32 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:03.516572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:03.517430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.517601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.517851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.518568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:03.520581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:03.596233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 875/1751] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:03 2022] POST /identity/v3/projects => generated 383 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:03.600692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:03.619881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1292e0a-1582-4dfd-aa29-2b05de7cda45 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:03.791646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:03.792418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.792705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.793753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.794513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:03.796835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:03.835018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 877/1752] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:03 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:03.842817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:03.860183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:03.862820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:03.868790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131fc1dc-9e00-48dd-8f18-2ea6fca63baa admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:03.889903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47711dd-000b-43d2-a1eb-94c4da6a1a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:03.890715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47711dd-000b-43d2-a1eb-94c4da6a1a47 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.891112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47711dd-000b-43d2-a1eb-94c4da6a1a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.891497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47711dd-000b-43d2-a1eb-94c4da6a1a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.892521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47711dd-000b-43d2-a1eb-94c4da6a1a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:03.897484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacfbef-e4d5-4766-8f2a-189d633b811f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:03.898428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47711dd-000b-43d2-a1eb-94c4da6a1a47 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:03.944942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:03.946523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.947142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.950607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 876/1753] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:03.951076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] PATH_INFO: `/v3/projects/708099e248894d6b89b7eca0e4cdc436` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.953006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73105b-ef53-4052-8d0f-7ef1ac25b2aa None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:03.955058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] RBAC: Authorizing `identity:delete_project(project_id=708099e248894d6b89b7eca0e4cdc436)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:03.957428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:03.959235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73105b-ef53-4052-8d0f-7ef1ac25b2aa None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:03.959398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c73105b-ef53-4052-8d0f-7ef1ac25b2aa None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:03.967067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 877/1754] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jul 7 10:26:03 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:26:03.971982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73105b-ef53-4052-8d0f-7ef1ac25b2aa None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:04.006283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c73105b-ef53-4052-8d0f-7ef1ac25b2aa None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:04.012170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f592ab-4b30-4a9d-bf23-59e039cdd38f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:04.013477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f592ab-4b30-4a9d-bf23-59e039cdd38f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.013764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f592ab-4b30-4a9d-bf23-59e039cdd38f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.014038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f592ab-4b30-4a9d-bf23-59e039cdd38f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.014773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f592ab-4b30-4a9d-bf23-59e039cdd38f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:04.016435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f592ab-4b30-4a9d-bf23-59e039cdd38f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:04.053396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 878/1755] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:04.055987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14f36c-304f-438a-8fa3-67c5e14fd519 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.056293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14f36c-304f-438a-8fa3-67c5e14fd519 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.056621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e14f36c-304f-438a-8fa3-67c5e14fd519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.064188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:04.104768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708099e248894d6b89b7eca0e4cdc436', 'request_id': 'req-63908972-e723-40e5-abd5-2527be05bcaf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.115846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708099e248894d6b89b7eca0e4cdc436', 'request_id': 'req-63908972-e723-40e5-abd5-2527be05bcaf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.137789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708099e248894d6b89b7eca0e4cdc436', 'request_id': 'req-63908972-e723-40e5-abd5-2527be05bcaf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.171462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] The token cache is being invalidate because project 708099e248894d6b89b7eca0e4cdc436 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:04.171803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.172393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.175263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9e14f36c-304f-438a-8fa3-67c5e14fd519 None None] MFA Rules not processed for user `61c23b3898c14a99bd1521644345d6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:04.177073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9e14f36c-304f-438a-8fa3-67c5e14fd519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:04.180374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 878/1756] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:03 2022] DELETE /identity/v3/projects/708099e248894d6b89b7eca0e4cdc436 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:04.181230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:04.186166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:04.342963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63908972-e723-40e5-abd5-2527be05bcaf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:04.353690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4bf83-1d3d-415a-8c1f-3b638937324a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:04.354437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4bf83-1d3d-415a-8c1f-3b638937324a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.354715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4bf83-1d3d-415a-8c1f-3b638937324a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.354998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4bf83-1d3d-415a-8c1f-3b638937324a admin admin] PATH_INFO: `/v3/projects/da915253297f4e839febb7ba5c2a894c/users/c4f2f8f310784648abea771da910262e/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.401548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4bf83-1d3d-415a-8c1f-3b638937324a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da915253297f4e839febb7ba5c2a894c, user_id=c4f2f8f310784648abea771da910262e, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:04.404606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4bf83-1d3d-415a-8c1f-3b638937324a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:04.427221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 879/1757] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:04 2022] PUT /identity/v3/projects/da915253297f4e839febb7ba5c2a894c/users/c4f2f8f310784648abea771da910262e/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:04.430279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.431551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.431908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.569251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] MFA Rules not processed for user `730e7d9b92c84bd5872bc0e221b616cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:04.581510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:04.623928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 879/1758] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 569 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:04.625517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.625966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.626353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.673946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:04.675363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:04.682759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 880/1759] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:04.686404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:04.689477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:04.751435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:04.824755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f142877-4856-4293-9dd7-3519d2d40f1b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:04.829979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:04.830858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.831133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.831409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] PATH_INFO: `/v3/projects/88295873db5d48bbb3f65410cda48aa8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.832664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88295873db5d48bbb3f65410cda48aa8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:04.839876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 880/1760] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:04.840032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:04.840123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:04.842381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:04.860437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0ca0d8-3070-458f-8e4a-c96718777689 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:04.879126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:04.880248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.880410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.880691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] PATH_INFO: `/v3/projects/26866bd48a1b436e983df989ea51bcbe` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.882251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26866bd48a1b436e983df989ea51bcbe)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:04.885053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:04.928135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88295873db5d48bbb3f65410cda48aa8', 'request_id': 'req-a0625648-17ff-412b-9aad-6d4d80c4a401'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.940933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88295873db5d48bbb3f65410cda48aa8', 'request_id': 'req-a0625648-17ff-412b-9aad-6d4d80c4a401'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.953707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88295873db5d48bbb3f65410cda48aa8', 'request_id': 'req-a0625648-17ff-412b-9aad-6d4d80c4a401'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.968434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] The token cache is being invalidate because project 88295873db5d48bbb3f65410cda48aa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:04.969045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.970060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a0625648-17ff-412b-9aad-6d4d80c4a401 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:04.972612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 881/1761] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:04 2022] DELETE /identity/v3/projects/88295873db5d48bbb3f65410cda48aa8 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:04.974190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:04.974728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:04.975309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:04.992512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:04.999507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26866bd48a1b436e983df989ea51bcbe', 'request_id': 'req-cb0aa0fd-3d53-45bf-96f8-d674266edd80'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.012548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26866bd48a1b436e983df989ea51bcbe', 'request_id': 'req-cb0aa0fd-3d53-45bf-96f8-d674266edd80'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.025443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26866bd48a1b436e983df989ea51bcbe', 'request_id': 'req-cb0aa0fd-3d53-45bf-96f8-d674266edd80'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.033180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] MFA Rules not processed for user `c4f2f8f310784648abea771da910262e`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:05.036523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.053035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] The token cache is being invalidate because project 26866bd48a1b436e983df989ea51bcbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:05.053344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.054178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.056108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 881/1762] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:04 2022] DELETE /identity/v3/projects/26866bd48a1b436e983df989ea51bcbe => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:05.056939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:05.062947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.135417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 882/1763] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:05.136978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:05.139869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.144105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0aa0fd-3d53-45bf-96f8-d674266edd80 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:05.149067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:05.150028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.150418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.150895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] PATH_INFO: `/v3/projects/48a8062587cb41da8711b1cc21e401ec` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.153215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48a8062587cb41da8711b1cc21e401ec)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:05.155489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:05.215692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:05.221471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853141e-304e-44b5-9ee3-a4510ebf38b6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:05.224568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48a8062587cb41da8711b1cc21e401ec', 'request_id': 'req-90943438-5aa8-48c6-a20d-96a56240bdf7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.240666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48a8062587cb41da8711b1cc21e401ec', 'request_id': 'req-90943438-5aa8-48c6-a20d-96a56240bdf7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.252064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48a8062587cb41da8711b1cc21e401ec', 'request_id': 'req-90943438-5aa8-48c6-a20d-96a56240bdf7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.266028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] The token cache is being invalidate because project 48a8062587cb41da8711b1cc21e401ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:05.266410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.267160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-90943438-5aa8-48c6-a20d-96a56240bdf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.269694 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 882/1764] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:05 2022] DELETE /identity/v3/projects/48a8062587cb41da8711b1cc21e401ec => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:05.274150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.274509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.274922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.294143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] MFA Rules not processed for user `730e7d9b92c84bd5872bc0e221b616cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:05.296023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.297201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26d8bf6-27f6-4568-9747-f09eb4dbb8dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:05.298164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d8bf6-27f6-4568-9747-f09eb4dbb8dc service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.298647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d8bf6-27f6-4568-9747-f09eb4dbb8dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.299149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26d8bf6-27f6-4568-9747-f09eb4dbb8dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.301817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c26d8bf6-27f6-4568-9747-f09eb4dbb8dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.316454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d8bf6-27f6-4568-9747-f09eb4dbb8dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:05.317899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26d8bf6-27f6-4568-9747-f09eb4dbb8dc service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:05.329478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 883/1765] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:05.331482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c671e3-58e8-4129-99c3-944fefa15171 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.331785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c671e3-58e8-4129-99c3-944fefa15171 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.332066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c671e3-58e8-4129-99c3-944fefa15171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.354716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a8c671e3-58e8-4129-99c3-944fefa15171 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:05.357704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a8c671e3-58e8-4129-99c3-944fefa15171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.403178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 883/1766] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:05.404223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:05.405640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.417624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:05.481303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 884/1767] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:05.481303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.481303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.481303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.510302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e51a2-2230-4ead-b67e-782fc5896a7f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:05.512810 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] MFA Rules not processed for user `247787cd745444f38953a64780d6ac9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:05.514036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.520771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:05.522026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.522499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.522956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] PATH_INFO: `/v3/users/c1a48833654c426282fe7a4fa6ea305b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.525181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1a48833654c426282fe7a4fa6ea305b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:05.527926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:05.580565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 885/1768] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:05.584802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:05.586555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.603009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2a2470-6bce-4d08-a16b-2a44a5a438c3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:05.608454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:05.609984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.609984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.610374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] PATH_INFO: `/v3/users/0a4c247a460e4740b824fb7a417760c8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.611530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a4c247a460e4740b824fb7a417760c8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:05.613710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:05.624123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1a48833654c426282fe7a4fa6ea305b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.639952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1a48833654c426282fe7a4fa6ea305b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.640602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1a48833654c426282fe7a4fa6ea305b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.659952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1a48833654c426282fe7a4fa6ea305b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.674200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1a48833654c426282fe7a4fa6ea305b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.685218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 884/1769] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:05 2022] DELETE /identity/v3/users/c1a48833654c426282fe7a4fa6ea305b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:05.685347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a4c247a460e4740b824fb7a417760c8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.686141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:05.687431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:05.688950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.712713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a4c247a460e4740b824fb7a417760c8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.713541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a4c247a460e4740b824fb7a417760c8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.724961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a4c247a460e4740b824fb7a417760c8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.743235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a4c247a460e4740b824fb7a417760c8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:05.757935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 886/1770] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:05 2022] DELETE /identity/v3/users/0a4c247a460e4740b824fb7a417760c8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:05.761330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:05.765137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:05.882502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b557cd1f-eb4a-4f45-9a85-426d16b54af2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:05.919896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21b3b9b-41e0-4a47-886c-b51fa83e10f9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:05.925440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:05.926816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:05.927251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:05.927678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] PATH_INFO: `/v3/users/839d13367e1649babf76876157e8a364` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:05.941519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=839d13367e1649babf76876157e8a364)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:05.947836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:06.097222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:06.124359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '839d13367e1649babf76876157e8a364'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.136853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '839d13367e1649babf76876157e8a364'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.137939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '839d13367e1649babf76876157e8a364'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.150610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '839d13367e1649babf76876157e8a364'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.168615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '839d13367e1649babf76876157e8a364'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.178714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 887/1771] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:05 2022] DELETE /identity/v3/users/839d13367e1649babf76876157e8a364 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:06.178714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:06.178714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.220111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:06.220806 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:06.221092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:06.221367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:06.222075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:06.224107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:06.236611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:06.273097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213d3bb-817a-48c7-b71c-35cde510c04b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:06.281709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:06.283453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:06.284744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:06.285204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:06.288980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.299109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 885/1772] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:05 2022] POST /identity/v3/projects => generated 381 bytes in 613 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:06.300476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:06.306595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.389177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:06.391955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:06.404001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 888/1773] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:06.419129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:06.423152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.424907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1787df7e-e152-4d98-bf65-f0f8cd254620 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:06.430846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:06.432937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:06.433227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:06.433543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:06.434350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:06.434667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:06.436382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:06.451088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 886/1774] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:06.451088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:06.454461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.618855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e0fdff-76aa-4ca1-9399-f89683a653ee service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:06.621419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0fd1d-a62f-43a0-ae88-d7c71b4ef5f3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:06.634051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:06.635037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:06.635314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:06.635589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] PATH_INFO: `/v3/projects/2130f287560041b5bcd75970b2adb316` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:06.644709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2130f287560041b5bcd75970b2adb316)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:06.645663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:06.646828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:06.647295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:06.647672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:06.648086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:06.650315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.679900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:06.681951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:06.710247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 887/1775] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:06.716573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:06.721640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.744566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab59f40-3cf0-421a-bfd0-3c7739e04778 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:06.757988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2130f287560041b5bcd75970b2adb316', 'request_id': 'req-a6506da2-af6b-4932-9d3e-2594c99a80fb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.774409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2130f287560041b5bcd75970b2adb316', 'request_id': 'req-a6506da2-af6b-4932-9d3e-2594c99a80fb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.796228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2130f287560041b5bcd75970b2adb316', 'request_id': 'req-a6506da2-af6b-4932-9d3e-2594c99a80fb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.838781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:06.839610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] The token cache is being invalidate because project 2130f287560041b5bcd75970b2adb316 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:06.840559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.841244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:06.843475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 889/1776] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:06 2022] DELETE /identity/v3/projects/2130f287560041b5bcd75970b2adb316 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:06.844033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:06.851270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.885527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:06.887074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:06.887455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:06.888018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:06.888834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:06.892853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:06.910981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:06.940643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 888/1777] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:06 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:06.943089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:06.948465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:06.980908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf778c7-b2ae-4ffb-b4ee-c40d349d2345 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:06.994700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:06.995755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:06.996185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:06.996572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] PATH_INFO: `/v3/projects/df3b29d6141b4a838c90695b6b98a6d0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:06.998569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=df3b29d6141b4a838c90695b6b98a6d0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.002329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.074860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6506da2-af6b-4932-9d3e-2594c99a80fb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.097293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31111974-3c71-4b2f-8663-1f5fa21e0ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.097987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111974-3c71-4b2f-8663-1f5fa21e0ccf admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.098375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111974-3c71-4b2f-8663-1f5fa21e0ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.098656 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31111974-3c71-4b2f-8663-1f5fa21e0ccf admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.099357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31111974-3c71-4b2f-8663-1f5fa21e0ccf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.108102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31111974-3c71-4b2f-8663-1f5fa21e0ccf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.114170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df3b29d6141b4a838c90695b6b98a6d0', 'request_id': 'req-827fcc32-a060-42ce-acce-307759dc9c8d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:07.128271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df3b29d6141b4a838c90695b6b98a6d0', 'request_id': 'req-827fcc32-a060-42ce-acce-307759dc9c8d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:07.144408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df3b29d6141b4a838c90695b6b98a6d0', 'request_id': 'req-827fcc32-a060-42ce-acce-307759dc9c8d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:07.190273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] The token cache is being invalidate because project df3b29d6141b4a838c90695b6b98a6d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:07.190857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:07.192263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:07.200623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 890/1779] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:06 2022] POST /identity/v3/projects => generated 373 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:07.201456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 889/1779] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:06 2022] DELETE /identity/v3/projects/df3b29d6141b4a838c90695b6b98a6d0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:07.202762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f975dd-cf8f-4c1b-b334-3e1590e1e562 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.203008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.204271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f975dd-cf8f-4c1b-b334-3e1590e1e562 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.204553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f975dd-cf8f-4c1b-b334-3e1590e1e562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.205388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:07.244609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-24f975dd-cf8f-4c1b-b334-3e1590e1e562 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:07.247537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24f975dd-cf8f-4c1b-b334-3e1590e1e562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:07.346286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827fcc32-a060-42ce-acce-307759dc9c8d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.388616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 891/1780] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:07.391615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.392265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.392736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.413200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:07.414817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:07.430143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 892/1781] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:07.431563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.432457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dd0229-dc23-44b8-a1ee-6183139d60ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.433871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd0229-dc23-44b8-a1ee-6183139d60ed admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.434314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:07.434672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd0229-dc23-44b8-a1ee-6183139d60ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.435024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dd0229-dc23-44b8-a1ee-6183139d60ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.435870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd0229-dc23-44b8-a1ee-6183139d60ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.437557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dd0229-dc23-44b8-a1ee-6183139d60ed admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.446216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:07.448872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bb692f-012a-43ab-bdae-642f9f37b960 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.455308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68796e83-54c2-40fc-aa10-eae5eed4e5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.456206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 890/1782] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:07.457087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68796e83-54c2-40fc-aa10-eae5eed4e5d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.457830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68796e83-54c2-40fc-aa10-eae5eed4e5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.458028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd92b-d31c-4845-af8f-386a710fc089 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.458388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68796e83-54c2-40fc-aa10-eae5eed4e5d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.458495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd92b-d31c-4845-af8f-386a710fc089 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.458864 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bfd92b-d31c-4845-af8f-386a710fc089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.459778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68796e83-54c2-40fc-aa10-eae5eed4e5d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.462885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68796e83-54c2-40fc-aa10-eae5eed4e5d5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.485988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-85bfd92b-d31c-4845-af8f-386a710fc089 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:07.488278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-85bfd92b-d31c-4845-af8f-386a710fc089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:07.494756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 893/1783] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:07 2022] POST /identity/v3/users => generated 287 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:07.500304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 891/1784] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:07.500682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a8f99-00f4-4e5a-ae92-0ec9196b93de None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.501858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd92b-d31c-4845-af8f-386a710fc089 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.502477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a8f99-00f4-4e5a-ae92-0ec9196b93de None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.502901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a8f99-00f4-4e5a-ae92-0ec9196b93de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.515086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bfd92b-d31c-4845-af8f-386a710fc089 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.520987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.522526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.523937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.524329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] PATH_INFO: `/v3/projects/cacd5ff80f714e6e8b67e74ff993636e/users/2e2216d1ae0c4942a9b9ba6cddad49c1/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.545421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cacd5ff80f714e6e8b67e74ff993636e, user_id=2e2216d1ae0c4942a9b9ba6cddad49c1, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.547985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d1a8f99-00f4-4e5a-ae92-0ec9196b93de None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:07.548854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.549208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d1a8f99-00f4-4e5a-ae92-0ec9196b93de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:07.571754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 894/1785] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:07.572706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a8f99-00f4-4e5a-ae92-0ec9196b93de None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.579221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 892/1786] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:07 2022] PUT /identity/v3/projects/cacd5ff80f714e6e8b67e74ff993636e/users/2e2216d1ae0c4942a9b9ba6cddad49c1/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:07.579221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.598740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a8f99-00f4-4e5a-ae92-0ec9196b93de None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.764821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1fb059-0c15-4004-a4d6-8b28356b764e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.774078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.774774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.775059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.775336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.776124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:07.776435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.778087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.791253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.794965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.795531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.796108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.797221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.800178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.806495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 893/1787] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:07.807376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.820097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 895/1788] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:07.822593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.835527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:07.842665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de82554-b85d-45ee-864e-7d1307cf900e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.854605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d717e6-dbce-4978-8cc6-3264fb1d19fe admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.879644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f3884-49d7-43f6-8102-cb0f257dafc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.879644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3884-49d7-43f6-8102-cb0f257dafc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.879644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3884-49d7-43f6-8102-cb0f257dafc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.879644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f3884-49d7-43f6-8102-cb0f257dafc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.880050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f3884-49d7-43f6-8102-cb0f257dafc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.880050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9f3884-49d7-43f6-8102-cb0f257dafc0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.891848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.893235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.894172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.894601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.895500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:07.895906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.897602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.914119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 896/1789] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:07.916156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea32f-a0a9-4d2b-8751-e403eb121db6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.916886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea32f-a0a9-4d2b-8751-e403eb121db6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.917416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fea32f-a0a9-4d2b-8751-e403eb121db6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.920368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 897/1790] 10.222.0.16 () {54 vars in 912 bytes} [Thu Jul 7 10:26:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:26:07.920591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea32f-a0a9-4d2b-8751-e403eb121db6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.922505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-89fea32f-a0a9-4d2b-8751-e403eb121db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:07.953881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fea32f-a0a9-4d2b-8751-e403eb121db6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.955826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 894/1791] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:07.957044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:07.961741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.963025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.963666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:07.963999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] PATH_INFO: `/v3/projects/4e8c7fc738f54d80bd58d5d32f2fe7d7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:07.966004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e8c7fc738f54d80bd58d5d32f2fe7d7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:07.968766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:07.982820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bb7460-c00a-4e47-91bb-119d603ade04 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:07.998798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:07.999483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:07.999835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.000075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.000783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.010574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:08.049005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e8c7fc738f54d80bd58d5d32f2fe7d7', 'request_id': 'req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:08.059977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 895/1792] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:07 2022] POST /identity/v3/projects => generated 409 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:08.061988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.073224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e8c7fc738f54d80bd58d5d32f2fe7d7', 'request_id': 'req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:08.086581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e8c7fc738f54d80bd58d5d32f2fe7d7', 'request_id': 'req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:08.102685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] The token cache is being invalidate because project 4e8c7fc738f54d80bd58d5d32f2fe7d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:08.103343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:08.104717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:08.107394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 898/1793] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:07 2022] DELETE /identity/v3/projects/4e8c7fc738f54d80bd58d5d32f2fe7d7 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:08.108637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.112590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.141284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5574d7f-594a-439f-aabe-7224321c074a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:08.148037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9271d8-7e8a-4faf-a248-a14b8d1ae01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:08.149114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9271d8-7e8a-4faf-a248-a14b8d1ae01b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.149450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9271d8-7e8a-4faf-a248-a14b8d1ae01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.149745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9271d8-7e8a-4faf-a248-a14b8d1ae01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.150686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9271d8-7e8a-4faf-a248-a14b8d1ae01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.153983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9271d8-7e8a-4faf-a248-a14b8d1ae01b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:08.179720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 896/1794] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:08.181648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.182032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.182416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.204812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] MFA Rules not processed for user `61c23b3898c14a99bd1521644345d6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:08.206357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.230836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:08.236946 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ae3098-fbfb-4cd4-8fca-65580c18bab0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:08.243780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:08.245022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.245401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.245798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] PATH_INFO: `/v3/projects/cacd5ff80f714e6e8b67e74ff993636e/users/2e2216d1ae0c4942a9b9ba6cddad49c1/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.248210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cacd5ff80f714e6e8b67e74ff993636e, user_id=2e2216d1ae0c4942a9b9ba6cddad49c1, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.251300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:08.284440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 899/1795] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:08 2022] PUT /identity/v3/projects/cacd5ff80f714e6e8b67e74ff993636e/users/2e2216d1ae0c4942a9b9ba6cddad49c1/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:08.288949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.297225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.316482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 897/1796] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:08.317625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.319195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.340628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cceea3-b83d-41cf-9ca8-345df288b819 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:08.389162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7545ed-9ff4-49cd-b6b6-49d139ee130f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:08.394393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999fd8f0-b946-4ae5-bab0-f6a372a4b436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:08.395126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd8f0-b946-4ae5-bab0-f6a372a4b436 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.395527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd8f0-b946-4ae5-bab0-f6a372a4b436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.395850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999fd8f0-b946-4ae5-bab0-f6a372a4b436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.396585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fd8f0-b946-4ae5-bab0-f6a372a4b436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.398625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999fd8f0-b946-4ae5-bab0-f6a372a4b436 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:08.415100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:08.438295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 900/1797] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:08 2022] POST /identity/v3/projects => generated 363 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:08.438295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.438295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.438295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.460255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] MFA Rules not processed for user `92b758b22f054dd991e8af99354a134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:08.461897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.469081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4da76a-fb43-44a8-91f1-0e185ade069b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:08.470166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4da76a-fb43-44a8-91f1-0e185ade069b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.470516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4da76a-fb43-44a8-91f1-0e185ade069b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.470806 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4da76a-fb43-44a8-91f1-0e185ade069b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.471524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4da76a-fb43-44a8-91f1-0e185ade069b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.473739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4da76a-fb43-44a8-91f1-0e185ade069b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:08.509646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 898/1798] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:08 2022] POST /identity/v3/users => generated 305 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:08.517306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640b08-540b-4997-86ff-987863b2fb2b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.517306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640b08-540b-4997-86ff-987863b2fb2b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.517560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4640b08-540b-4997-86ff-987863b2fb2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.545678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c4640b08-540b-4997-86ff-987863b2fb2b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:08.548796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4640b08-540b-4997-86ff-987863b2fb2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.571860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 901/1799] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:08.572984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.578249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.689743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 899/1800] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:08.689743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f138-e8af-455e-9956-b5d8626d145c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.689743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f138-e8af-455e-9956-b5d8626d145c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.689743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a0f138-e8af-455e-9956-b5d8626d145c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.726303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cb69e-dd8c-4f65-94e8-f334e3aa1e35 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:08.737887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f9a0f138-e8af-455e-9956-b5d8626d145c None None] MFA Rules not processed for user `2e2216d1ae0c4942a9b9ba6cddad49c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:08.739141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f9a0f138-e8af-455e-9956-b5d8626d145c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.740631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:08.742229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.742745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.743217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] PATH_INFO: `/v3/projects/74957572d45347ebb85fcde024a49cbd/users/2de8229355264aeeb03f5cb1c505153b/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.782511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=74957572d45347ebb85fcde024a49cbd, user_id=2de8229355264aeeb03f5cb1c505153b, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.786847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:08.811133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 902/1801] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:08 2022] PUT /identity/v3/projects/74957572d45347ebb85fcde024a49cbd/users/2de8229355264aeeb03f5cb1c505153b/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:08.812105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.813836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:08.838617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:08.876811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 900/1802] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:08.878369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f138-e8af-455e-9956-b5d8626d145c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.891641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a0f138-e8af-455e-9956-b5d8626d145c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:08.917070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8607c03-a92a-4e27-83f4-d9c52c8e99f0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:08.921858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:08.922730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.923074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.923556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.931657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.933336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:08.963070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 903/1803] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:08.964020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:08.988664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfff579-5ed5-47d0-9684-7cf752742e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:08.989320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfff579-5ed5-47d0-9684-7cf752742e72 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:08.989589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfff579-5ed5-47d0-9684-7cf752742e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:08.989870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfff579-5ed5-47d0-9684-7cf752742e72 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:08.990794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfff579-5ed5-47d0-9684-7cf752742e72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:08.997291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfff579-5ed5-47d0-9684-7cf752742e72 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.037423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:09.048015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 901/1804] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:08 2022] POST /identity/v3/users => generated 282 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:09.049613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dad56-e954-46af-a458-055c718883a0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.049975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dad56-e954-46af-a458-055c718883a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.050869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834dad56-e954-46af-a458-055c718883a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.076052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-834dad56-e954-46af-a458-055c718883a0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:09.077594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-834dad56-e954-46af-a458-055c718883a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:09.088359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 902/1805] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:09.090665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3354a-b3ec-4e1f-8fbd-cbc4a562972f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.091114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3354a-b3ec-4e1f-8fbd-cbc4a562972f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.091759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e3354a-b3ec-4e1f-8fbd-cbc4a562972f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.128421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256036d-0aa5-4cae-a2a4-e014d24a4ab9 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.140610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.140924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.141273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c4e3354a-b3ec-4e1f-8fbd-cbc4a562972f None None] MFA Rules not processed for user `2de8229355264aeeb03f5cb1c505153b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:09.141414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.142186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.142925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.143140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c4e3354a-b3ec-4e1f-8fbd-cbc4a562972f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:09.144867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.167713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 904/1806] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:08 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:09.168802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.178449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:09.291045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a067f05b-78e6-4eea-a47b-37f9a4f319cb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.301570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 903/1807] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:09.302852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3354a-b3ec-4e1f-8fbd-cbc4a562972f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.312590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.313296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.316051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.316342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.323326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e3354a-b3ec-4e1f-8fbd-cbc4a562972f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.328971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.329849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.330730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.330962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.332032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.334313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.349363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 904/1808] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:09.350254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.371610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637282f-d8a4-4417-97b0-1be388a9d11e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.380623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.381325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.382220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.382512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.383571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:09.383931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.385855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.422667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 905/1809] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:09.422667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.468422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22c085-a113-42bb-b55e-c1d33f5a9b44 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.478503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.481054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.482253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.482662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.484153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.484452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.486931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.488949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.498007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 905/1810] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:09.499372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.506294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 906/1811] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:09.507180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.522459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734ee66-08d4-4ad5-8732-e3cf6998add6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.542940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e448532-a68d-48db-80aa-b152b54b0571 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.553641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.554438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.555061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.555349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.556128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.557821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.566056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.566749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.567038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.567401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.568313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.578286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.588248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 907/1812] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:09.589199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.612915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54792ddf-e648-4cc5-bbd0-e8c193003c59 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.632116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.632945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.633304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.633675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.634604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.637198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.659677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 906/1813] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:09 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:09.666427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.677441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:09.708732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 908/1814] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:09 2022] POST /identity/v3/projects => generated 399 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:09.709702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.815736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:09.847145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48214cda-cf80-49cd-aabe-0eecaf2ff0ab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.858721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.861090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.861739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.862033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] PATH_INFO: `/v3/projects/24e4caaa20604a38854e0c038ece9459/users/09e021c1cd054b5eb141621c92973df9/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.872224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e8824f-3945-44ab-a64c-c3e9bac56e98 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:09.884428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:09.885360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:09.885644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:09.885921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] PATH_INFO: `/v3/projects/15dbaeafcb484b70b6b495233c832d03/users/0026f7455ce94e3292d41d9f53abad4f/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:09.909204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=24e4caaa20604a38854e0c038ece9459, user_id=09e021c1cd054b5eb141621c92973df9, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.913429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.921588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15dbaeafcb484b70b6b495233c832d03, user_id=0026f7455ce94e3292d41d9f53abad4f, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:09.926870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:09.943894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 907/1815] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:09 2022] PUT /identity/v3/projects/24e4caaa20604a38854e0c038ece9459/users/09e021c1cd054b5eb141621c92973df9/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:09.943894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:09.950235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 909/1816] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:09 2022] PUT /identity/v3/projects/15dbaeafcb484b70b6b495233c832d03/users/0026f7455ce94e3292d41d9f53abad4f/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:09.952019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:10.082323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b048d1-e20d-4cec-b4dd-8c631feb8c5b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:10.134417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f32913b-de75-4520-8d56-830681cbf057 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:10.159293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:10.161386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.161681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.162545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.163280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:10.173138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:10.221847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 910/1817] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:09 2022] POST /identity/v3/users => generated 292 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:10.222947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:10.237152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e0469-8e72-4d7e-b7c2-93fbf7b13f87 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:10.239034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfe957e-95f9-491d-99f8-1ca677d74b31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:10.240321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe957e-95f9-491d-99f8-1ca677d74b31 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.240655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe957e-95f9-491d-99f8-1ca677d74b31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.240939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe957e-95f9-491d-99f8-1ca677d74b31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.245466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9dfe957e-95f9-491d-99f8-1ca677d74b31 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:10.252098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b5994-4304-4e38-ab06-0006edfa57ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:10.252802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5994-4304-4e38-ab06-0006edfa57ba admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.253088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5994-4304-4e38-ab06-0006edfa57ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.253689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b5994-4304-4e38-ab06-0006edfa57ba admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.257084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b5994-4304-4e38-ab06-0006edfa57ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:10.263680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b5994-4304-4e38-ab06-0006edfa57ba admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:10.329800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 911/1818] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:10 2022] POST /identity/v3/users => generated 300 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:10.332182 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159e085-c770-4ab5-ac31-aef74a51c1be None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.332681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159e085-c770-4ab5-ac31-aef74a51c1be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.333227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2159e085-c770-4ab5-ac31-aef74a51c1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.376677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe957e-95f9-491d-99f8-1ca677d74b31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:10.378260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfe957e-95f9-491d-99f8-1ca677d74b31 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:10.379613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2159e085-c770-4ab5-ac31-aef74a51c1be None None] MFA Rules not processed for user `09e021c1cd054b5eb141621c92973df9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:10.381518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2159e085-c770-4ab5-ac31-aef74a51c1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:10.412122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 908/1819] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:10.414176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3656731-debc-40ff-b291-ba21fc66a732 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.423334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3656731-debc-40ff-b291-ba21fc66a732 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.423954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3656731-debc-40ff-b291-ba21fc66a732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.472563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:10.490421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f3656731-debc-40ff-b291-ba21fc66a732 None None] MFA Rules not processed for user `0026f7455ce94e3292d41d9f53abad4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:10.492149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f3656731-debc-40ff-b291-ba21fc66a732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:10.558938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:10.572316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 912/1820] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:10.574378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de4fff-7951-4f33-90b1-c0bbb0431f56 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.574555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de4fff-7951-4f33-90b1-c0bbb0431f56 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.574839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3de4fff-7951-4f33-90b1-c0bbb0431f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.627476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d3de4fff-7951-4f33-90b1-c0bbb0431f56 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:10.629085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d3de4fff-7951-4f33-90b1-c0bbb0431f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:10.701273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 909/1821] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:10.702308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3656731-debc-40ff-b291-ba21fc66a732 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:10.729442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 913/1822] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:26:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:10.741043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3656731-debc-40ff-b291-ba21fc66a732 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:10.744970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de4fff-7951-4f33-90b1-c0bbb0431f56 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:10.760041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:10.760927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.761869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.762141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.762876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:10.765478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:10.796808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3de4fff-7951-4f33-90b1-c0bbb0431f56 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:10.810241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 910/1823] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:10.811449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:10.815490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:10.816400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.816752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.817038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.817784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:10.828695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:10.831631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474fc1de-e4d9-4c9b-9074-646ab226a8cb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:10.848631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:10.849662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.850337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.851968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.853031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:10.863233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 914/1824] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:10.864161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:10.864299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:10.893179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520170bc-5cee-4b30-a984-6cd93a3b306a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:10.906060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:10.907066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:10.908013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:10.908161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:10.909173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:10.912918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:10.962515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 915/1825] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:10 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:10.963495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:10.980813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 911/1826] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:10 2022] POST /identity/v3/projects => generated 407 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:10.982068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.084551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6751e8-14cb-45fc-ad46-e62349c76e4e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.115600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d300ae1-6f27-4b80-98de-3e69510df4bf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.125624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.126403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.126752 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.127096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.127991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.130352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.145837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 912/1827] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:10 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:11.146556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.160611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474cb38-d1b6-49ac-ac79-f121eadd180a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.168393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.170317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.170670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.171008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.172421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.175707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.210312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 913/1828] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:11 2022] POST /identity/v3/users => generated 289 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:11.211378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.229050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:11.238488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.240437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dcad55-2039-4a34-a58a-6a1a923a96f3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.241450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.242045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.242564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] PATH_INFO: `/v3/projects/bc44f0d8a86f4142a75869f9b800d275/users/7fd67b73bd4247ecad888626a87d04c5/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.249015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.251235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.251522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.251836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] PATH_INFO: `/v3/users/c4f2f8f310784648abea771da910262e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.253370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4f2f8f310784648abea771da910262e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.255385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.278371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc44f0d8a86f4142a75869f9b800d275, user_id=7fd67b73bd4247ecad888626a87d04c5, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.281143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.300981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 916/1829] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:10 2022] PUT /identity/v3/projects/bc44f0d8a86f4142a75869f9b800d275/users/7fd67b73bd4247ecad888626a87d04c5/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:11.302164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.383885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4f2f8f310784648abea771da910262e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:11.402986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4f2f8f310784648abea771da910262e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:11.403848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4f2f8f310784648abea771da910262e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:11.421231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4f2f8f310784648abea771da910262e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:11.431452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:11.463286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4f2f8f310784648abea771da910262e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:11.466192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9eb8e7-2662-4e76-85b2-137e753db5ef admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.474062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.474814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.475097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.475385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.476211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.478873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.485204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 914/1830] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:11 2022] DELETE /identity/v3/users/c4f2f8f310784648abea771da910262e => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:11.487011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.489754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:11.518889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 917/1831] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:11 2022] POST /identity/v3/users => generated 310 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:11.520007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.521881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:11.539538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1673e23-563b-422c-8914-b2f1e4ea3a6a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.590725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e82cd72-1d23-4cb1-860a-bf69f287e598 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.604925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.606649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.607599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.609001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.611472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:11.616668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.617754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.618053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.618334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] PATH_INFO: `/v3/projects/d24fa74971ad457fb63df382979d3216/users/c8abf334b24844328513e4fc9c5b04c1/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.642154 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d24fa74971ad457fb63df382979d3216, user_id=c8abf334b24844328513e4fc9c5b04c1, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.646121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.683589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 918/1832] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:11 2022] PUT /identity/v3/projects/d24fa74971ad457fb63df382979d3216/users/c8abf334b24844328513e4fc9c5b04c1/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:11.684670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.686320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:11.715433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.717897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.737836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 915/1833] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:26:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:26:11.738861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.751115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:11.785345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:11.802800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53922c85-087d-4e9a-ade8-a1fa1fbac2b3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.828030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25af530-be92-4819-9533-6fbca6cf95b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.832037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af530-be92-4819-9533-6fbca6cf95b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.836280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29d94de-ed3d-4b1e-aecd-04f66c11bb2d service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:11.836777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af530-be92-4819-9533-6fbca6cf95b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.836777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25af530-be92-4819-9533-6fbca6cf95b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.842762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25af530-be92-4819-9533-6fbca6cf95b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.851468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25af530-be92-4819-9533-6fbca6cf95b9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.857503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:11.861256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.862449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.862749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.867413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:11.874348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:11.891425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 919/1834] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:11.892956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0b914-632c-439b-be2a-4969a8d99710 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:11.893247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0b914-632c-439b-be2a-4969a8d99710 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:11.893874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc0b914-632c-439b-be2a-4969a8d99710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:11.919939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 916/1835] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:11.938801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:11.941747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:11.956069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-afc0b914-632c-439b-be2a-4969a8d99710 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:11.962875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-afc0b914-632c-439b-be2a-4969a8d99710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:12.004964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36968947-a20c-470f-9d64-fdd683588bb5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:12.014663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 920/1836] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:12.018745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d0091-37e6-469a-8ffa-a6cfd7ae1061 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.019552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d0091-37e6-469a-8ffa-a6cfd7ae1061 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.020801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01d0091-37e6-469a-8ffa-a6cfd7ae1061 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.025733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2bbeb24-43ce-4f72-981c-5561dcbfd4af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:12.029109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbeb24-43ce-4f72-981c-5561dcbfd4af admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.029386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbeb24-43ce-4f72-981c-5561dcbfd4af admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.029668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bbeb24-43ce-4f72-981c-5561dcbfd4af admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.030452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbeb24-43ce-4f72-981c-5561dcbfd4af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:12.033343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2bbeb24-43ce-4f72-981c-5561dcbfd4af admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:12.070542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 917/1837] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:12.070542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce01c88-87ad-48e4-bd4c-73db2b51ce10 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.070542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce01c88-87ad-48e4-bd4c-73db2b51ce10 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.072667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce01c88-87ad-48e4-bd4c-73db2b51ce10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.101412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f01d0091-37e6-469a-8ffa-a6cfd7ae1061 None None] MFA Rules not processed for user `61c23b3898c14a99bd1521644345d6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:12.114428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f01d0091-37e6-469a-8ffa-a6cfd7ae1061 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:12.134997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0ce01c88-87ad-48e4-bd4c-73db2b51ce10 None None] MFA Rules not processed for user `c8abf334b24844328513e4fc9c5b04c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:12.137333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0ce01c88-87ad-48e4-bd4c-73db2b51ce10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:12.292602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 921/1838] 10.222.0.58 () {58 vars in 1015 bytes} [Thu Jul 7 10:26:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:12.294024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d0091-37e6-469a-8ffa-a6cfd7ae1061 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:12.310348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01d0091-37e6-469a-8ffa-a6cfd7ae1061 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:12.316293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:12.318095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.318764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.319274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] PATH_INFO: `/v3/projects/c707fa00981347a9b5239ae39be990d8/users/38963dfc33904bdfa67e3f2559f9d9bd/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.352527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:12.376204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c707fa00981347a9b5239ae39be990d8, user_id=38963dfc33904bdfa67e3f2559f9d9bd, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:12.377971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 918/1839] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:12.384908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce01c88-87ad-48e4-bd4c-73db2b51ce10 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:12.385064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:12.413060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce01c88-87ad-48e4-bd4c-73db2b51ce10 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:12.418243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 922/1840] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:12 2022] PUT /identity/v3/projects/c707fa00981347a9b5239ae39be990d8/users/38963dfc33904bdfa67e3f2559f9d9bd/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:12.422899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:12.546076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:12.547803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.554633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.554633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] PATH_INFO: `/v3/projects/bc44f0d8a86f4142a75869f9b800d275/users/7fd67b73bd4247ecad888626a87d04c5/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.557306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc44f0d8a86f4142a75869f9b800d275, user_id=7fd67b73bd4247ecad888626a87d04c5, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:12.564026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:12.586070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 919/1841] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:12 2022] PUT /identity/v3/projects/bc44f0d8a86f4142a75869f9b800d275/users/7fd67b73bd4247ecad888626a87d04c5/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:12.587856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:12.595834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:12.611222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c1c88-6f3b-4d74-9c02-212a614b80e1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:12.752382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e52bd-8a38-4a6c-b5f1-7e7e9ad24b63 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:12.758111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:12.759093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.759328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.759732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] PATH_INFO: `/v3/projects/20af2c18f0ff4a7a8a70290cc1c64f2e/users/9d2d57fe427546a8b7aeb6ff4f46e0a8/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.765820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:12.766523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.768074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.768347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.788116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:12.789582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:12.801246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=20af2c18f0ff4a7a8a70290cc1c64f2e, user_id=9d2d57fe427546a8b7aeb6ff4f46e0a8, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:12.805456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:12.813821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 920/1842] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:12.815416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:12.822519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:12.835139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 923/1843] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:12 2022] PUT /identity/v3/projects/20af2c18f0ff4a7a8a70290cc1c64f2e/users/9d2d57fe427546a8b7aeb6ff4f46e0a8/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:12.835139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:12.836620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede3f3e8-4062-43fa-b7ce-8d626d1eb1da service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:12.969805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:12.970485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:12.970768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:12.971082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:12.971807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:12.977026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d7e787-b287-41aa-986c-4da533540d6d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:12.979110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.004332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b859c-3b59-4ed3-b10b-4d4bc28ff056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:13.004332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b859c-3b59-4ed3-b10b-4d4bc28ff056 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.004332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b859c-3b59-4ed3-b10b-4d4bc28ff056 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.004332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b859c-3b59-4ed3-b10b-4d4bc28ff056 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.004828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b859c-3b59-4ed3-b10b-4d4bc28ff056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:13.004828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b859c-3b59-4ed3-b10b-4d4bc28ff056 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.031255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 924/1844] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:12 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:13.032725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2131aa4-ec41-4df8-807f-e5637816600e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.033312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2131aa4-ec41-4df8-807f-e5637816600e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.033513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2131aa4-ec41-4df8-807f-e5637816600e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.064465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 921/1845] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:12 2022] POST /identity/v3/projects => generated 401 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:13.065268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:13.089203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d2131aa4-ec41-4df8-807f-e5637816600e None None] MFA Rules not processed for user `7fd67b73bd4247ecad888626a87d04c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:13.094900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d2131aa4-ec41-4df8-807f-e5637816600e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:13.151523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:13.259274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d8574c-8a31-47ab-8a30-1f6584c67034 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:13.278329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 925/1846] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:13.278550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:13.278662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2131aa4-ec41-4df8-807f-e5637816600e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:13.278777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.278777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.278777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.278777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:13.301351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.332442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 922/1847] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:13.333732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:13.362077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9fafa-6423-4c13-9e9e-8657015a7af1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:13.372869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:13.374452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.375002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.375606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] PATH_INFO: `/v3/projects/c707fa00981347a9b5239ae39be990d8/users/38963dfc33904bdfa67e3f2559f9d9bd/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.379131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c707fa00981347a9b5239ae39be990d8, user_id=38963dfc33904bdfa67e3f2559f9d9bd, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:13.399446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2131aa4-ec41-4df8-807f-e5637816600e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:13.404422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:13.405100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.405388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.405667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.406141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.423497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:13.426391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.448511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 923/1848] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:13 2022] PUT /identity/v3/projects/c707fa00981347a9b5239ae39be990d8/users/38963dfc33904bdfa67e3f2559f9d9bd/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:13.448511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:13.452368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 926/1849] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:13.453499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:13.455298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:13.512156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:13.596046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943eaea4-8620-492b-b6bb-a17f1d3c2bb9 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:13.625522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778aa9e6-008d-48bf-89a5-b64cd801dce3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:13.680767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:13.681610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.681780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.682077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.686523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:13.688799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.729478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:13.730297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.730660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.731008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.737730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 924/1850] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:13 2022] POST /identity/v3/users => generated 307 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:13.739595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:13.768321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b667d44c-5089-41b2-897a-922b2daf1046 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:13.786674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ea06d-2ccf-40fd-8371-b7154299659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:13.789848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea06d-2ccf-40fd-8371-b7154299659d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.790242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea06d-2ccf-40fd-8371-b7154299659d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.790530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ea06d-2ccf-40fd-8371-b7154299659d admin admin] PATH_INFO: `/v3/projects/20af2c18f0ff4a7a8a70290cc1c64f2e/users/9d2d57fe427546a8b7aeb6ff4f46e0a8/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.797551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ea06d-2ccf-40fd-8371-b7154299659d admin admin] RBAC: Authorizing `identity:create_grant(project_id=20af2c18f0ff4a7a8a70290cc1c64f2e, user_id=9d2d57fe427546a8b7aeb6ff4f46e0a8, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:13.807406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690ea06d-2ccf-40fd-8371-b7154299659d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.847882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 925/1851] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:13 2022] PUT /identity/v3/projects/20af2c18f0ff4a7a8a70290cc1c64f2e/users/9d2d57fe427546a8b7aeb6ff4f46e0a8/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:13.854062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5ff03-f1fd-4ff4-bfa9-ad3beb46e353 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:13.854652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5ff03-f1fd-4ff4-bfa9-ad3beb46e353 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:13.855163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b5ff03-f1fd-4ff4-bfa9-ad3beb46e353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:13.903112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e2b5ff03-f1fd-4ff4-bfa9-ad3beb46e353 None None] MFA Rules not processed for user `38963dfc33904bdfa67e3f2559f9d9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:13.904450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e2b5ff03-f1fd-4ff4-bfa9-ad3beb46e353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:13.917835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:13.918784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:13.934425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 927/1852] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:13 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:13.935414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:13.974816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab52023-2f34-4544-8f87-bd1d126b709d service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:14.109126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b0ee77-e180-4d4e-89d3-251cf8cf3672 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:14.109883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ee77-e180-4d4e-89d3-251cf8cf3672 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.110114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ee77-e180-4d4e-89d3-251cf8cf3672 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.110529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b0ee77-e180-4d4e-89d3-251cf8cf3672 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:14.111345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0ee77-e180-4d4e-89d3-251cf8cf3672 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:14.113080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b0ee77-e180-4d4e-89d3-251cf8cf3672 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:14.140621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 928/1853] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:14.142214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbb23f-2817-4494-a158-14f6f6453ad4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.143534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbb23f-2817-4494-a158-14f6f6453ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.143534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbb23f-2817-4494-a158-14f6f6453ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:14.193075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 926/1854] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:14.193075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5ff03-f1fd-4ff4-bfa9-ad3beb46e353 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:14.204599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:14.217221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b5ff03-f1fd-4ff4-bfa9-ad3beb46e353 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:14.235707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:14.236846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.237286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.237789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] PATH_INFO: `/v3/projects/d7b29cd99cc54c999d35309060a572ab/users/eea0c9383ae14fc8944d3265721e8a8c/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:14.259871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-31fbb23f-2817-4494-a158-14f6f6453ad4 None None] MFA Rules not processed for user `9d2d57fe427546a8b7aeb6ff4f46e0a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:14.264975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-31fbb23f-2817-4494-a158-14f6f6453ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:14.287175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b29cd99cc54c999d35309060a572ab, user_id=eea0c9383ae14fc8944d3265721e8a8c, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:14.299759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:14.332276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:14.340622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 927/1855] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:14 2022] PUT /identity/v3/projects/d7b29cd99cc54c999d35309060a572ab/users/eea0c9383ae14fc8944d3265721e8a8c/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:14.341757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:14.532627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3805a5-f604-4f30-82ea-a276dd1191df admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:14.540778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:14.543379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.543679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 929/1856] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 402 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:14.546310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.546586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbb23f-2817-4494-a158-14f6f6453ad4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:14.547062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:14.584837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbb23f-2817-4494-a158-14f6f6453ad4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:14.743764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:14.744748 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.745039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.745381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:14.746839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:14.750176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:14.750622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:14.752559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:14.766609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 928/1857] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:14.768373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:14.770120 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:14.786804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 930/1858] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:14.788215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e708ba-3d2a-4dd6-adcb-02ac34d93e25 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:14.789978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:14.794299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:14.795306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.795865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.796204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:14.828539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0ada7a-01c3-4ea1-918f-03f4964b3d20 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:14.848575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f427-5e9a-4cce-980f-6b227b79e8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:14.849560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f427-5e9a-4cce-980f-6b227b79e8eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.850226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f427-5e9a-4cce-980f-6b227b79e8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.850777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f427-5e9a-4cce-980f-6b227b79e8eb admin admin] PATH_INFO: `/v3/projects/d7b29cd99cc54c999d35309060a572ab/users/eea0c9383ae14fc8944d3265721e8a8c/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:14.858019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61f427-5e9a-4cce-980f-6b227b79e8eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b29cd99cc54c999d35309060a572ab, user_id=eea0c9383ae14fc8944d3265721e8a8c, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:14.861129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa61f427-5e9a-4cce-980f-6b227b79e8eb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:14.918352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 931/1859] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:14 2022] PUT /identity/v3/projects/d7b29cd99cc54c999d35309060a572ab/users/eea0c9383ae14fc8944d3265721e8a8c/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:14.921467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76047ca-f470-4ad7-936e-9df9aa83060a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:14.922607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76047ca-f470-4ad7-936e-9df9aa83060a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:14.923575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76047ca-f470-4ad7-936e-9df9aa83060a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:15.016538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:15.018117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:15.038707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d76047ca-f470-4ad7-936e-9df9aa83060a None None] MFA Rules not processed for user `eea0c9383ae14fc8944d3265721e8a8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:15.041835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d76047ca-f470-4ad7-936e-9df9aa83060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:15.064385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 929/1860] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:14 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:15.135410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:15.146147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffc699-e500-4f56-98bc-eb4f33c3df8c service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:15.255175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:15.298334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 932/1861] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:14 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:15.359059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:15.361486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:15.361486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:15.361486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:15.364342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:15.520743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76047ca-f470-4ad7-936e-9df9aa83060a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:15.564807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:15.590391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:15.592928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:15.631374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 930/1862] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:15 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:15.632895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:15.647509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c807a0-8d5f-4763-a455-d7b4dc208691 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:15.654283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76047ca-f470-4ad7-936e-9df9aa83060a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:15.664586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:15.666862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:15.667248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:15.668220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:15.669539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:15.670459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:15.670725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:15.670983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] PATH_INFO: `/v3/users/2de8229355264aeeb03f5cb1c505153b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:15.672460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=2de8229355264aeeb03f5cb1c505153b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:15.674489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:15.720193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:15.721547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:15.741166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 931/1863] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:15 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:15.777831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2de8229355264aeeb03f5cb1c505153b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:15.789961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2de8229355264aeeb03f5cb1c505153b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:15.790843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2de8229355264aeeb03f5cb1c505153b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:15.813729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2de8229355264aeeb03f5cb1c505153b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:15.835356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d0e5504c-6669-4b10-b485-735a00f696ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2de8229355264aeeb03f5cb1c505153b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:15.852566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:15.853893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 933/1864] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:15 2022] DELETE /identity/v3/users/2de8229355264aeeb03f5cb1c505153b => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:15.855756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:15.856877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:15.857381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:15.861862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781b8556-7334-43d8-b07f-724ef17e3bb3 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:15.920873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:15.929808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:16.065798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8295e2-853a-48ee-b2d4-b5007bad4fc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:16.069281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8295e2-853a-48ee-b2d4-b5007bad4fc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:16.070133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8295e2-853a-48ee-b2d4-b5007bad4fc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:16.070702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8295e2-853a-48ee-b2d4-b5007bad4fc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:16.075897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-af8295e2-853a-48ee-b2d4-b5007bad4fc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:16.111178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 934/1865] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:16.114627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:16.116221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:16.149524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b5687-0003-403f-87a0-cf5ebb4c7b81 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:16.165195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:16.165878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:16.166456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:16.166816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:16.172046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:16.261928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:16.315794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:16.317446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:16.335829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 935/1866] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:16 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:16.336592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:16.338763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:16.352302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8295e2-853a-48ee-b2d4-b5007bad4fc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:16.360740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8295e2-853a-48ee-b2d4-b5007bad4fc5 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:16.400949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 932/1867] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:16.407256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc4a26-7b07-4867-a655-a790698f2daa None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:16.408532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc4a26-7b07-4867-a655-a790698f2daa None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:16.409413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fc4a26-7b07-4867-a655-a790698f2daa None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:16.416024 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 933/1868] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:26:16 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:26:16.483649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbdcf3e-b05d-4ff9-b240-39e5f3d75121 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:16.507205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:16.508810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:16.509106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:16.509389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:16.538040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:16.541592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:16.553929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:16.553929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:16.553929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:16.585811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 936/1869] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:16.601511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:16.602929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:16.646257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:16.776033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 934/1870] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:26:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:16.990433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:17.000600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:17.029915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff41ce-42f6-42fd-93b1-b81d3306b122 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:17.045070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:17.047736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:17.048577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:17.049278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] PATH_INFO: `/v3/projects/da915253297f4e839febb7ba5c2a894c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:17.052370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da915253297f4e839febb7ba5c2a894c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:17.054573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:17.176300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da915253297f4e839febb7ba5c2a894c', 'request_id': 'req-45ef091c-dcd4-4eaa-8d9f-96eec7206698'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.193182 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da915253297f4e839febb7ba5c2a894c', 'request_id': 'req-45ef091c-dcd4-4eaa-8d9f-96eec7206698'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.204910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da915253297f4e839febb7ba5c2a894c', 'request_id': 'req-45ef091c-dcd4-4eaa-8d9f-96eec7206698'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.214182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:17.216194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:17.221750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] The token cache is being invalidate because project da915253297f4e839febb7ba5c2a894c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:17.222301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.223350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.232231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 937/1871] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:16 2022] DELETE /identity/v3/projects/da915253297f4e839febb7ba5c2a894c => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:17.234100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:17.238109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:17.263947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fe94c5-cc57-4b45-b69f-3ab556874667 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:17.382148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef091c-dcd4-4eaa-8d9f-96eec7206698 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:17.404834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:17.410896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:17.414831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:17.415785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] PATH_INFO: `/v3/users/2e2216d1ae0c4942a9b9ba6cddad49c1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:17.419238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e2216d1ae0c4942a9b9ba6cddad49c1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:17.427614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:17.439223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:17.498973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72713e31-ba2c-43b2-b9af-9d0549bd2bf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:17.499703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713e31-ba2c-43b2-b9af-9d0549bd2bf6 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:17.500029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713e31-ba2c-43b2-b9af-9d0549bd2bf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:17.500352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72713e31-ba2c-43b2-b9af-9d0549bd2bf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:17.525574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e2216d1ae0c4942a9b9ba6cddad49c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.532460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-72713e31-ba2c-43b2-b9af-9d0549bd2bf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:17.541029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e2216d1ae0c4942a9b9ba6cddad49c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.542830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e2216d1ae0c4942a9b9ba6cddad49c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.563654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72713e31-ba2c-43b2-b9af-9d0549bd2bf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:17.564210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72713e31-ba2c-43b2-b9af-9d0549bd2bf6 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:17.581436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e2216d1ae0c4942a9b9ba6cddad49c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.592291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e2216d1ae0c4942a9b9ba6cddad49c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.600990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 938/1872] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:17 2022] DELETE /identity/v3/users/2e2216d1ae0c4942a9b9ba6cddad49c1 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:17.602995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:17.603861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 935/1873] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:17.604465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:17.727893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adc7cf6-15cc-4303-9538-1448ff0a6208 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:17.734828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:17.735849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:17.736941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:17.736941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] PATH_INFO: `/v3/users/0026f7455ce94e3292d41d9f53abad4f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:17.738562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0026f7455ce94e3292d41d9f53abad4f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:17.741257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:17.858811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0026f7455ce94e3292d41d9f53abad4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.870136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0026f7455ce94e3292d41d9f53abad4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.871416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0026f7455ce94e3292d41d9f53abad4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.897710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0026f7455ce94e3292d41d9f53abad4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.913412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0026f7455ce94e3292d41d9f53abad4f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:17.918950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:17.931725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 939/1874] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:17 2022] DELETE /identity/v3/users/0026f7455ce94e3292d41d9f53abad4f => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:17.936442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:17.937038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:17.937915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:17.973943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:17.981011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:18.075793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 936/1875] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:18.102975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:18.105424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:18.218468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e66327b-21da-4b33-9e8f-571a83aeab55 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:18.224293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:18.225706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:18.227587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:18.227949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:18.252045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:18.254391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:18.270803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 940/1876] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:18.765989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:18.768844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:18.801364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d5c409-a0e9-46d9-a647-3c74d568be70 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:18.815297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:18.816237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:18.818457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:18.818751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] PATH_INFO: `/v3/projects/74957572d45347ebb85fcde024a49cbd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:18.822669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] RBAC: Authorizing `identity:delete_project(project_id=74957572d45347ebb85fcde024a49cbd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:18.835825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:18.950617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:19.010727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '74957572d45347ebb85fcde024a49cbd', 'request_id': 'req-b2bb6212-d3a2-4e61-aebb-606945a2809b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.050435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '74957572d45347ebb85fcde024a49cbd', 'request_id': 'req-b2bb6212-d3a2-4e61-aebb-606945a2809b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.069302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '74957572d45347ebb85fcde024a49cbd', 'request_id': 'req-b2bb6212-d3a2-4e61-aebb-606945a2809b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.120381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] The token cache is being invalidate because project 74957572d45347ebb85fcde024a49cbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:19.122086 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.124625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b2bb6212-d3a2-4e61-aebb-606945a2809b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.146486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 937/1877] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:18 2022] DELETE /identity/v3/projects/74957572d45347ebb85fcde024a49cbd => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:19.147125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:19.158060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91eca24b-89d8-4b56-9c3c-e9a608ce355b service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:19.347880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:19.351909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:19.352305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:19.352649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] PATH_INFO: `/v3/users/7fd67b73bd4247ecad888626a87d04c5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:19.359217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fd67b73bd4247ecad888626a87d04c5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:19.361484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:19.493239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fd67b73bd4247ecad888626a87d04c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.516187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fd67b73bd4247ecad888626a87d04c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.516941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fd67b73bd4247ecad888626a87d04c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.539993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fd67b73bd4247ecad888626a87d04c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.556522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-93aa0fbd-f8e6-4c60-8605-e9b2a097ed7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fd67b73bd4247ecad888626a87d04c5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:19.574787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 941/1878] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:19 2022] DELETE /identity/v3/users/7fd67b73bd4247ecad888626a87d04c5 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:19.864608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee244b-04d0-4032-9dc6-174eed942ff5 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:19.865089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee244b-04d0-4032-9dc6-174eed942ff5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:19.865443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ee244b-04d0-4032-9dc6-174eed942ff5 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:19.867432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 938/1879] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:26:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:26:19.893434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:19.893739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:19.894171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:19.923864 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:19.925093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:19.951831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:20.044835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee244b-04d0-4032-9dc6-174eed942ff5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:20.047309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a2ee244b-04d0-4032-9dc6-174eed942ff5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:20.055820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 942/1880] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:26:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:20.175416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ee244b-04d0-4032-9dc6-174eed942ff5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:20.181048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:20.185719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:20.185979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:20.186383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] PATH_INFO: `/v3/projects/15dbaeafcb484b70b6b495233c832d03` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:20.188442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] RBAC: Authorizing `identity:delete_project(project_id=15dbaeafcb484b70b6b495233c832d03)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:20.191728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:20.238334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:20.240984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:20.268826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4986534-cfd6-4025-a651-e79ef16d6a97 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:20.295466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15dbaeafcb484b70b6b495233c832d03', 'request_id': 'req-635ada4c-569c-4f8a-8a4b-2143f402fd8d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.303972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15dbaeafcb484b70b6b495233c832d03', 'request_id': 'req-635ada4c-569c-4f8a-8a4b-2143f402fd8d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.326744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15dbaeafcb484b70b6b495233c832d03', 'request_id': 'req-635ada4c-569c-4f8a-8a4b-2143f402fd8d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.344750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] The token cache is being invalidate because project 15dbaeafcb484b70b6b495233c832d03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:20.345141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.346029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.349791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 939/1881] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:20 2022] DELETE /identity/v3/projects/15dbaeafcb484b70b6b495233c832d03 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:20.350917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:20.359622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:20.428737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:20.433965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:20.434257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:20.434603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] PATH_INFO: `/v3/projects/bc44f0d8a86f4142a75869f9b800d275` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:20.436477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc44f0d8a86f4142a75869f9b800d275)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:20.438415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:20.445773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:20.514228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ada4c-569c-4f8a-8a4b-2143f402fd8d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:20.525583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:20.526280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:20.526545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:20.527397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:20.540517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:20.575557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:20.577293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:20.613219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 940/1882] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:20.619169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:20.619969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:20.658785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774dc67-9245-4d53-adbb-03cfa89ae21c service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:20.665765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc44f0d8a86f4142a75869f9b800d275', 'request_id': 'req-23c4785b-18b1-4fe2-bd27-75773b67d343'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.687430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc44f0d8a86f4142a75869f9b800d275', 'request_id': 'req-23c4785b-18b1-4fe2-bd27-75773b67d343'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.711229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc44f0d8a86f4142a75869f9b800d275', 'request_id': 'req-23c4785b-18b1-4fe2-bd27-75773b67d343'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.740891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] The token cache is being invalidate because project bc44f0d8a86f4142a75869f9b800d275 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:20.741322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.742112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:20.744785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 943/1883] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:20 2022] DELETE /identity/v3/projects/bc44f0d8a86f4142a75869f9b800d275 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:20.746616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:20.749136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:20.827957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:20.877792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c4785b-18b1-4fe2-bd27-75773b67d343 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:20.878631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:20.880348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:20.881083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:20.881863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] PATH_INFO: `/v3/users/38963dfc33904bdfa67e3f2559f9d9bd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:20.884729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38963dfc33904bdfa67e3f2559f9d9bd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:20.887989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:20.891430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:20.892492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:20.892879 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:20.893263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] PATH_INFO: `/v3/projects/cacd5ff80f714e6e8b67e74ff993636e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:20.894637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=cacd5ff80f714e6e8b67e74ff993636e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:20.897990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:20.992318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38963dfc33904bdfa67e3f2559f9d9bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.007701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cacd5ff80f714e6e8b67e74ff993636e', 'request_id': 'req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.011696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38963dfc33904bdfa67e3f2559f9d9bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.012585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38963dfc33904bdfa67e3f2559f9d9bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.025085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cacd5ff80f714e6e8b67e74ff993636e', 'request_id': 'req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.025574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38963dfc33904bdfa67e3f2559f9d9bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.037717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cacd5ff80f714e6e8b67e74ff993636e', 'request_id': 'req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.057520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e799914d-2e34-415b-9027-a5986f9a0275 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38963dfc33904bdfa67e3f2559f9d9bd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.073174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 941/1884] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:20 2022] DELETE /identity/v3/users/38963dfc33904bdfa67e3f2559f9d9bd => generated 0 bytes in 459 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:21.075726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77832bd-1aec-48bb-8e6f-d900f83739c6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.076040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77832bd-1aec-48bb-8e6f-d900f83739c6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.077444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77832bd-1aec-48bb-8e6f-d900f83739c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.079573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] The token cache is being invalidate because project cacd5ff80f714e6e8b67e74ff993636e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:21.085121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.086508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9d0f7d31-6590-479e-bb79-e92b1b1f3e2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.089061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 944/1885] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:20 2022] DELETE /identity/v3/projects/cacd5ff80f714e6e8b67e74ff993636e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:21.095082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538083-6cef-49c2-bc17-2fe02c330a29 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.095888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538083-6cef-49c2-bc17-2fe02c330a29 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.096277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd538083-6cef-49c2-bc17-2fe02c330a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.122854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c77832bd-1aec-48bb-8e6f-d900f83739c6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:21.126562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c77832bd-1aec-48bb-8e6f-d900f83739c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.135864 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cd538083-6cef-49c2-bc17-2fe02c330a29 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:21.137237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cd538083-6cef-49c2-bc17-2fe02c330a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.241353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:21.255607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 945/1886] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:21.265318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53644e24-4e0d-4b40-b649-aa0caf94d9f9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.265619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53644e24-4e0d-4b40-b649-aa0caf94d9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.267032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53644e24-4e0d-4b40-b649-aa0caf94d9f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.275702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 942/1887] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:21.278160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06262154-a160-4920-98c9-9f2981ea4796 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.278458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06262154-a160-4920-98c9-9f2981ea4796 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.278749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06262154-a160-4920-98c9-9f2981ea4796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.294264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-53644e24-4e0d-4b40-b649-aa0caf94d9f9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:21.295500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-53644e24-4e0d-4b40-b649-aa0caf94d9f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.307881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 946/1888] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:21.310309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53644e24-4e0d-4b40-b649-aa0caf94d9f9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.320214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53644e24-4e0d-4b40-b649-aa0caf94d9f9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.333950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.334722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.335235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.335493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.336469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:21.336586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-06262154-a160-4920-98c9-9f2981ea4796 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:21.336680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.337798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-06262154-a160-4920-98c9-9f2981ea4796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.338158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.353129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 943/1889] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:21.357992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06262154-a160-4920-98c9-9f2981ea4796 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.369376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 947/1890] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:21.373008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06262154-a160-4920-98c9-9f2981ea4796 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.374527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.381404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.382720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.383005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.383278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.384045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:21.384352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.386260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.397269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a582f1-4eeb-4d4d-b5c7-203b2d977ee1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.403137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.403974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.404639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.405005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.405782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.408419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.414029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 944/1891] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:21.420149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.440008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8d5ab-4d70-4fbf-be88-5490888131d2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.447263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 948/1892] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:21.451048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.454738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.455709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.456042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.456417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.457383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.459627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.499101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:21.505487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 945/1893] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/projects => generated 375 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:21.506474 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.510233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.549142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d5ff3b-db94-4fa4-98b7-c8b4f64898e8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.582682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336be09-6e7b-4395-8d8c-0a913976fb1a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.587776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.588678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.588982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.589899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] PATH_INFO: `/v3/projects/c707fa00981347a9b5239ae39be990d8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.591101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c707fa00981347a9b5239ae39be990d8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.593043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.605439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.606126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.606415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.606694 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.607393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.609641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.634283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 949/1894] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:21.635103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.645976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560327f-8b61-4121-82df-f7dcb0b914f7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.646279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c707fa00981347a9b5239ae39be990d8', 'request_id': 'req-ac74490a-05d7-408f-bc6f-d5ce8f09b509'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.654309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c707fa00981347a9b5239ae39be990d8', 'request_id': 'req-ac74490a-05d7-408f-bc6f-d5ce8f09b509'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.663586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c707fa00981347a9b5239ae39be990d8', 'request_id': 'req-ac74490a-05d7-408f-bc6f-d5ce8f09b509'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.679348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] The token cache is being invalidate because project c707fa00981347a9b5239ae39be990d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:21.679737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.680477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:21.682333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 946/1895] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:21 2022] DELETE /identity/v3/projects/c707fa00981347a9b5239ae39be990d8 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:21.683227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.684622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.710533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61c453e-1675-4b95-9f8d-b1c2017d8966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.711332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c453e-1675-4b95-9f8d-b1c2017d8966 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.711838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c453e-1675-4b95-9f8d-b1c2017d8966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.712291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61c453e-1675-4b95-9f8d-b1c2017d8966 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.713185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c453e-1675-4b95-9f8d-b1c2017d8966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.715678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61c453e-1675-4b95-9f8d-b1c2017d8966 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.737364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:21.739557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 950/1896] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/users => generated 288 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:21.740968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285bba-4899-4cf9-bf4d-aeab9dc9a264 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.741262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285bba-4899-4cf9-bf4d-aeab9dc9a264 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.742207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70285bba-4899-4cf9-bf4d-aeab9dc9a264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.759586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac74490a-05d7-408f-bc6f-d5ce8f09b509 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.765511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70285bba-4899-4cf9-bf4d-aeab9dc9a264 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:21.765655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.766188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.766473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.766767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.767476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.767693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70285bba-4899-4cf9-bf4d-aeab9dc9a264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.769119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.780749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 947/1897] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:21.781771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.783862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.787873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 951/1898] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:21.788838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70285bba-4899-4cf9-bf4d-aeab9dc9a264 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.801527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70285bba-4899-4cf9-bf4d-aeab9dc9a264 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.806192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970dafff-0684-4ee0-a4f8-5b35bdc1ebb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.806792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45316eec-4987-4cd3-9143-96975a990eb0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.806898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970dafff-0684-4ee0-a4f8-5b35bdc1ebb6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.807160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970dafff-0684-4ee0-a4f8-5b35bdc1ebb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.807444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970dafff-0684-4ee0-a4f8-5b35bdc1ebb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.808240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970dafff-0684-4ee0-a4f8-5b35bdc1ebb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.809835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970dafff-0684-4ee0-a4f8-5b35bdc1ebb6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.812426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.813284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.813657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.814038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.815313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.817058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.820028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 952/1899] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:21.821285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df866c-f95b-45c0-8925-299c7943dc68 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.821575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df866c-f95b-45c0-8925-299c7943dc68 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.821853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55df866c-f95b-45c0-8925-299c7943dc68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.831480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 948/1900] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:21.831480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.839910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40720e3b-7423-45b4-9551-5fe12bb64dd4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.840777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-55df866c-f95b-45c0-8925-299c7943dc68 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:21.841979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-55df866c-f95b-45c0-8925-299c7943dc68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.844535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.845518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.845821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.846169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] PATH_INFO: `/v3/projects/f7a8ad7892a64e55a74abd18a3d79450/users/c4a986882f1d4d4b8d64cb828ee65616/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.850092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 953/1901] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:21.850987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df866c-f95b-45c0-8925-299c7943dc68 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.862623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7a8ad7892a64e55a74abd18a3d79450, user_id=c4a986882f1d4d4b8d64cb828ee65616, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.863262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55df866c-f95b-45c0-8925-299c7943dc68 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.865503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.868359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8fb2b0-6a49-4917-94cd-87375990fca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.869536 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fb2b0-6a49-4917-94cd-87375990fca9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.869820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fb2b0-6a49-4917-94cd-87375990fca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.870233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8fb2b0-6a49-4917-94cd-87375990fca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.871155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fb2b0-6a49-4917-94cd-87375990fca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.872927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8fb2b0-6a49-4917-94cd-87375990fca9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.880421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 949/1902] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:21 2022] PUT /identity/v3/projects/f7a8ad7892a64e55a74abd18a3d79450/users/c4a986882f1d4d4b8d64cb828ee65616/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:21.881274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.884370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 954/1903] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:21 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:21.890971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217885e-e7c6-4bb2-a450-dcbffb537bc3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.891270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217885e-e7c6-4bb2-a450-dcbffb537bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.891565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217885e-e7c6-4bb2-a450-dcbffb537bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.928032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0217885e-e7c6-4bb2-a450-dcbffb537bc3 None None] MFA Rules not processed for user `c4a986882f1d4d4b8d64cb828ee65616`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:21.929272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0217885e-e7c6-4bb2-a450-dcbffb537bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:21.940946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:21.956602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce96629-a38e-4bbd-8e55-5571ea1c90ec admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.961370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.961977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.962254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.962526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.963275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:21.963582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:21.965780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:21.977420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 950/1904] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:21.978411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:21.988141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87dd6f-25e4-4f86-a01c-932beb996085 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:21.992749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:21.993604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:21.993887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:21.994164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] PATH_INFO: `/v3/projects/59bfb77cec7c4192909af3515a730817/users/614857eb4afe4b3da58d083731a1b683/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:21.996107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 955/1905] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:21.996695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217885e-e7c6-4bb2-a450-dcbffb537bc3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.006933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217885e-e7c6-4bb2-a450-dcbffb537bc3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.012010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a23607-1832-48c6-a29a-649e4493a5f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.012657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a23607-1832-48c6-a29a-649e4493a5f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.012753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59bfb77cec7c4192909af3515a730817, user_id=614857eb4afe4b3da58d083731a1b683, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.012942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a23607-1832-48c6-a29a-649e4493a5f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.013220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a23607-1832-48c6-a29a-649e4493a5f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.013898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a23607-1832-48c6-a29a-649e4493a5f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.015407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.015788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a23607-1832-48c6-a29a-649e4493a5f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.029502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 951/1906] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:21 2022] PUT /identity/v3/projects/59bfb77cec7c4192909af3515a730817/users/614857eb4afe4b3da58d083731a1b683/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:22.030630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.056150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 956/1907] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:21 2022] POST /identity/v3/projects => generated 381 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:22.056150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5171c-b147-4d5c-9ff8-8401f89705af None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.056150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5171c-b147-4d5c-9ff8-8401f89705af None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.056150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5171c-b147-4d5c-9ff8-8401f89705af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.081112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bbc5171c-b147-4d5c-9ff8-8401f89705af None None] MFA Rules not processed for user `614857eb4afe4b3da58d083731a1b683`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:22.082413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bbc5171c-b147-4d5c-9ff8-8401f89705af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:22.093309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9001d43-cf1b-42f9-958a-f071da235d42 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.111961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:22.162749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 957/1908] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:22.162749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5171c-b147-4d5c-9ff8-8401f89705af None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.164975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.165679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.165980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.166254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.166950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.168933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.179926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5171c-b147-4d5c-9ff8-8401f89705af None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.187016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.187893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.188349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.189790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.190730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.192717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.197750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 952/1909] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:22 2022] POST /identity/v3/projects => generated 381 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:22.199303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.216625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 958/1910] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:22 2022] POST /identity/v3/users => generated 291 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:22.219638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.237097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90162266-8a96-48d0-9c1b-68ddd8d533f1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.314178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e9f994-9bd6-4306-8316-ceeb6327d03a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.327618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.328699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.329149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.329593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.330876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.333730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.335876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.336577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.336865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.337149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.337887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.339913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.349337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:22.365872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 959/1911] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:22 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:22.367017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.385966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 953/1912] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:22 2022] POST /identity/v3/projects => generated 375 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:22.388022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.392227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1e6a4f-8440-4018-9a7d-8031ee9ad0e2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.463770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95dde4-605b-4da6-a4b8-720559ce8838 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.468954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.469556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.469837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.470108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.470824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.472581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.473127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.473938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.474105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.474414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.475181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.476812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.487052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 954/1913] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:22.488620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.499339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 960/1914] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:22.499339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.509606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6171b8a9-5a77-4001-accd-621c08f4e68f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.510459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c928ce-62a0-411b-8b40-723109d8057a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.514738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.515427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.515715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.516026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.516742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.518690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.522524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.523445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.523791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.524053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] PATH_INFO: `/v3/projects/0f66a00b0bd14545abf17edc37ee7bf8/users/70f4104c4d6642ae8ade42c669b4d643/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.553814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 955/1915] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:22 2022] POST /identity/v3/users => generated 294 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:22.554010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f66a00b0bd14545abf17edc37ee7bf8, user_id=70f4104c4d6642ae8ade42c669b4d643, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.554010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.554421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.579230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ba96c9-d9a1-4339-a63d-e604f6f2a7ef admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.588821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 961/1916] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:22 2022] PUT /identity/v3/projects/0f66a00b0bd14545abf17edc37ee7bf8/users/70f4104c4d6642ae8ade42c669b4d643/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:22.590396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.592843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:22.702358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5e9f1-327a-4167-9736-07f37c461d41 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.714946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.715969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.716583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.716988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.718177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.720348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.726506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.727176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.727467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.727766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.728508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.731341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.737169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 962/1917] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:22.738681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.743681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 956/1918] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:22.744621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.757386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f87379-e54b-42b9-acfc-a42e4e9ae21e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.762418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.762751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f5aee-7c65-4774-88c8-df2a50b6f5ae admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.763360 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.763664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.763961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] PATH_INFO: `/v3/projects/79d7b3ca71a94af3833f39cf98e233b8/users/f583a83f0d974512855096dffc5914c0/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.776284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.777364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.777839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.778136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.778963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.782716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.783958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d7b3ca71a94af3833f39cf98e233b8, user_id=f583a83f0d974512855096dffc5914c0, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.786992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.807943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 963/1919] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:22.807943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.808187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 957/1920] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:22 2022] PUT /identity/v3/projects/79d7b3ca71a94af3833f39cf98e233b8/users/f583a83f0d974512855096dffc5914c0/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:22.813549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:22.842054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ccb6a-2e9d-46dd-893c-27c4f39c6e4f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.912750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:22.925505 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.926850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.928002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.928657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] PATH_INFO: `/v3/projects/81ce412c59424901bc392bccfa701c18/users/e473ef0d1201403b9627efe2faaba3b9/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.930582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c87640-4b73-4ca2-b99c-8cbefc312602 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:22.940296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e435eaa-777c-4a28-b54e-89b9f6781142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:22.941212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e435eaa-777c-4a28-b54e-89b9f6781142 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.941496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e435eaa-777c-4a28-b54e-89b9f6781142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.941779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e435eaa-777c-4a28-b54e-89b9f6781142 admin admin] PATH_INFO: `/v3/projects/0f66a00b0bd14545abf17edc37ee7bf8/users/70f4104c4d6642ae8ade42c669b4d643/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.949744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81ce412c59424901bc392bccfa701c18, user_id=e473ef0d1201403b9627efe2faaba3b9, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.953042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.957500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e435eaa-777c-4a28-b54e-89b9f6781142 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f66a00b0bd14545abf17edc37ee7bf8, user_id=70f4104c4d6642ae8ade42c669b4d643, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:22.960139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e435eaa-777c-4a28-b54e-89b9f6781142 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:22.988767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 964/1922] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:22 2022] PUT /identity/v3/projects/81ce412c59424901bc392bccfa701c18/users/e473ef0d1201403b9627efe2faaba3b9/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:22.989779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 958/1922] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:22 2022] PUT /identity/v3/projects/0f66a00b0bd14545abf17edc37ee7bf8/users/70f4104c4d6642ae8ade42c669b4d643/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:22.991493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5305-1427-4927-9363-3e3e28d95f4c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:22.991811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5305-1427-4927-9363-3e3e28d95f4c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:22.992108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d5305-1427-4927-9363-3e3e28d95f4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:22.998855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:23.044600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f75d5305-1427-4927-9363-3e3e28d95f4c None None] MFA Rules not processed for user `70f4104c4d6642ae8ade42c669b4d643`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:23.047154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f75d5305-1427-4927-9363-3e3e28d95f4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.104524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aa519b-7396-46b2-ac72-319406196dc5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:23.111641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fea2b9-e6ba-4bbe-9311-7a3046afdcdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:23.112959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea2b9-e6ba-4bbe-9311-7a3046afdcdf admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.113247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea2b9-e6ba-4bbe-9311-7a3046afdcdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.113523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fea2b9-e6ba-4bbe-9311-7a3046afdcdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.114262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fea2b9-e6ba-4bbe-9311-7a3046afdcdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:23.116079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fea2b9-e6ba-4bbe-9311-7a3046afdcdf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:23.133166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 965/1923] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:22 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:23.134887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.135388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.135998 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.174267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] MFA Rules not processed for user `e473ef0d1201403b9627efe2faaba3b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:23.175721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.195955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 959/1924] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:22 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:23.196912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d5305-1427-4927-9363-3e3e28d95f4c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:23.202770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:23.210572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d5305-1427-4927-9363-3e3e28d95f4c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:23.219587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:23.220771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.221236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.221521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] PATH_INFO: `/v3/projects/79d7b3ca71a94af3833f39cf98e233b8/users/f583a83f0d974512855096dffc5914c0/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.224132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d7b3ca71a94af3833f39cf98e233b8, user_id=f583a83f0d974512855096dffc5914c0, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:23.227163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:23.239662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 960/1925] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:23 2022] PUT /identity/v3/projects/79d7b3ca71a94af3833f39cf98e233b8/users/f583a83f0d974512855096dffc5914c0/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:23.246715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:23.246715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.306868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 966/1926] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:23.310161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:23.312977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.354192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072c0a8-39e3-44f1-908f-7f2effd80a1b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:23.364098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f42eea0-07ec-43f8-8c92-20ee1029f1d8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:23.371413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd8ee9-03ab-4460-85ba-7e0cdaff2f5c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:23.372537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8ee9-03ab-4460-85ba-7e0cdaff2f5c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.372974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8ee9-03ab-4460-85ba-7e0cdaff2f5c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.374056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd8ee9-03ab-4460-85ba-7e0cdaff2f5c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.452442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:23.454658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.455467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.456044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] PATH_INFO: `/v3/users/c8abf334b24844328513e4fc9c5b04c1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.457534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8abf334b24844328513e4fc9c5b04c1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:23.460701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:23.462583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:23.488997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8ee9-03ab-4460-85ba-7e0cdaff2f5c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:23.490744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd8ee9-03ab-4460-85ba-7e0cdaff2f5c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:23.506767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 961/1927] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:23.508286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fd26b-7ee3-4201-8356-4b0d19786e14 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.518295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fd26b-7ee3-4201-8356-4b0d19786e14 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.518727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3fd26b-7ee3-4201-8356-4b0d19786e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.583053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ca3fd26b-7ee3-4201-8356-4b0d19786e14 None None] MFA Rules not processed for user `f583a83f0d974512855096dffc5914c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:23.583053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ca3fd26b-7ee3-4201-8356-4b0d19786e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.588283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8abf334b24844328513e4fc9c5b04c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:23.606378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8abf334b24844328513e4fc9c5b04c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:23.607293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8abf334b24844328513e4fc9c5b04c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:23.629584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8abf334b24844328513e4fc9c5b04c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:23.645096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8abf334b24844328513e4fc9c5b04c1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:23.663303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 967/1928] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:23 2022] DELETE /identity/v3/users/c8abf334b24844328513e4fc9c5b04c1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:23.664469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:23.668137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.753159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 962/1929] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:23.755171 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.755643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.756040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.794785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:23.796526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.807393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 963/1930] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:23.808435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:23.813443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b72e9aa-7ef3-42b8-8e90-e1dfada18710 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:23.815124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.824472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:23.825162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.825424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.825703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:23.826403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:23.830588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:23.870244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 968/1931] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:23 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:23.871411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:23.875135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:23.877031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:23.936919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2529b2-196e-4e28-8c2b-a2b5921b5c5b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:23.982124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953d773-4b73-425b-85b0-4b1cfd26f3a2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:23.989122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:23.990249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:23.990743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:23.991322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.029280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:24.065603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:24.066917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.067309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.067731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.070488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:24.116847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.118944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.134589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 969/1932] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:24.135899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.165835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983250f8-1d14-4cef-859b-42b7ad6948b2 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:24.173320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:24.175282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.175720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.176406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.177605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.179885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.225516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 970/1933] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:24 2022] POST /identity/v3/users => generated 297 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:24.229457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.237499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.239694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.250456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0346110-e5ff-410a-bd71-ea92eb19a863 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:24.262887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 964/1934] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:24.269423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:24.270455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.270844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.271225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.272358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.274036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.316421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 971/1935] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:24.321385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.342357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313901b6-8ec8-45c2-85a7-2d92d051601a service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:24.359464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:24.363173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.363586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.365137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] PATH_INFO: `/v3/projects/536b7f3bfe3044219f04ddf93aa5f6a6/users/07b67e22df0c41778f0ed0b659bf2da8/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.413844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536b7f3bfe3044219f04ddf93aa5f6a6, user_id=07b67e22df0c41778f0ed0b659bf2da8, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.426674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.473785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 965/1936] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:24 2022] PUT /identity/v3/projects/536b7f3bfe3044219f04ddf93aa5f6a6/users/07b67e22df0c41778f0ed0b659bf2da8/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:24.475830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.491032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7abf7-3323-45c3-a07f-b52516070947 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:24.590089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:24.591321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.591617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.592220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.593197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.595471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.627338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 972/1937] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:24.632480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.649796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecfe450-76a4-4c95-969b-c6b6ac4df1a3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:24.664199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f519396e-91c9-4767-a50e-3b6e85998d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:24.665514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519396e-91c9-4767-a50e-3b6e85998d56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.665952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519396e-91c9-4767-a50e-3b6e85998d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.680979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f519396e-91c9-4767-a50e-3b6e85998d56 admin admin] PATH_INFO: `/v3/projects/536b7f3bfe3044219f04ddf93aa5f6a6/users/07b67e22df0c41778f0ed0b659bf2da8/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.696092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519396e-91c9-4767-a50e-3b6e85998d56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=536b7f3bfe3044219f04ddf93aa5f6a6, user_id=07b67e22df0c41778f0ed0b659bf2da8, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.701477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f519396e-91c9-4767-a50e-3b6e85998d56 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.711509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.716133 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:24.737246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b8ced7-8798-41f5-bc57-ded9d295ceae admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:24.756791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 966/1938] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:24 2022] PUT /identity/v3/projects/536b7f3bfe3044219f04ddf93aa5f6a6/users/07b67e22df0c41778f0ed0b659bf2da8/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:24.768867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8477-88c9-435a-894a-da32e5d485ff None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.769111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8477-88c9-435a-894a-da32e5d485ff None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.769579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa8477-88c9-435a-894a-da32e5d485ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.842075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:24.843238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:24.843624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:24.844070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] PATH_INFO: `/v3/projects/d24fa74971ad457fb63df382979d3216` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:24.845672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d24fa74971ad457fb63df382979d3216)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:24.848275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:24.849332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-96fa8477-88c9-435a-894a-da32e5d485ff None None] MFA Rules not processed for user `07b67e22df0c41778f0ed0b659bf2da8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:24.852468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-96fa8477-88c9-435a-894a-da32e5d485ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:24.898878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:24.928784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:24.935764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd24fa74971ad457fb63df382979d3216', 'request_id': 'req-b40b7e27-090c-4b6b-bce9-9b4050534396'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:24.947607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd24fa74971ad457fb63df382979d3216', 'request_id': 'req-b40b7e27-090c-4b6b-bce9-9b4050534396'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:24.956773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd24fa74971ad457fb63df382979d3216', 'request_id': 'req-b40b7e27-090c-4b6b-bce9-9b4050534396'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:24.966707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 967/1939] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:24 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:24.973072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] The token cache is being invalidate because project d24fa74971ad457fb63df382979d3216 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:24.973638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:24.974389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:24.976267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 973/1940] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:24 2022] DELETE /identity/v3/projects/d24fa74971ad457fb63df382979d3216 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:24.980514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8477-88c9-435a-894a-da32e5d485ff None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.983059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:24.984668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:24.992318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa8477-88c9-435a-894a-da32e5d485ff None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:25.045690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b7e27-090c-4b6b-bce9-9b4050534396 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:25.049545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:25.050200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:25.050485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:25.050762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:25.052579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:25.052813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:25.053710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:25.053988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:25.054265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] PATH_INFO: `/v3/users/eea0c9383ae14fc8944d3265721e8a8c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:25.055434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea0c9383ae14fc8944d3265721e8a8c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:25.057487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:25.112881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea0c9383ae14fc8944d3265721e8a8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.121815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea0c9383ae14fc8944d3265721e8a8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.122576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea0c9383ae14fc8944d3265721e8a8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.130796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea0c9383ae14fc8944d3265721e8a8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.132825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:25.134473 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:25.147765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 974/1941] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:24 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:25.164111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea0c9383ae14fc8944d3265721e8a8c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.171181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 968/1942] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:24 2022] DELETE /identity/v3/users/eea0c9383ae14fc8944d3265721e8a8c => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:25.573390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:25.584882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:25.586183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2541b0-090d-4a4c-86ff-620d4cd86bce service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:25.587127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:25.720011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b67e22df0c41778f0ed0b659bf2da8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '536b7f3bfe3044219f04ddf93aa5f6a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:25.721009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:25.721395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:25.721960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:25.723047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:25.723412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:25.725306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:25.730686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e44efc-c555-4ea2-a407-a894daf1c48d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:25.735415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:25.740252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:25.746412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:25.746412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:25.748809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] PATH_INFO: `/v3/projects/d7b29cd99cc54c999d35309060a572ab` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:25.750655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b29cd99cc54c999d35309060a572ab)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:25.753644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 975/1943] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:25.754579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:25.757738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:25.762671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:25.764608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:25.772564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f58d9-09ca-432f-9893-7440c689b00d tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:25.780024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b67e22df0c41778f0ed0b659bf2da8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '536b7f3bfe3044219f04ddf93aa5f6a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:25.781544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:25.781782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:25.782672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:25.783788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:25.786580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:25.837558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 976/1944] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:25 2022] POST /identity/v3/projects => generated 353 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:25.843152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:25.860821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b29cd99cc54c999d35309060a572ab', 'request_id': 'req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.867500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b29cd99cc54c999d35309060a572ab', 'request_id': 'req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.876274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b29cd99cc54c999d35309060a572ab', 'request_id': 'req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.888963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] The token cache is being invalidate because project d7b29cd99cc54c999d35309060a572ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:25.889276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.889914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e159397a-7e98-40f5-b61d-b3cd9f5cf0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:25.891794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 969/1945] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:25 2022] DELETE /identity/v3/projects/d7b29cd99cc54c999d35309060a572ab => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:25.897175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:25.897772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:25.898551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:25.925732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9d83d0-8d9c-4710-bda6-1decd537ae23 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:25.933307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:25.934537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.008477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b52940-9d0a-4d9a-acd2-51893a14bb39 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b67e22df0c41778f0ed0b659bf2da8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '536b7f3bfe3044219f04ddf93aa5f6a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.009250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52940-9d0a-4d9a-acd2-51893a14bb39 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.009544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52940-9d0a-4d9a-acd2-51893a14bb39 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.009937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b52940-9d0a-4d9a-acd2-51893a14bb39 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.011073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b52940-9d0a-4d9a-acd2-51893a14bb39 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:26.012281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b52940-9d0a-4d9a-acd2-51893a14bb39 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.014331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b52940-9d0a-4d9a-acd2-51893a14bb39 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.035078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 977/1946] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:26.064321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 970/1947] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:26.066285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c2eb4-705f-457a-a704-9c03879ebb7c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.066711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c2eb4-705f-457a-a704-9c03879ebb7c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.067072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522c2eb4-705f-457a-a704-9c03879ebb7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.108905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-522c2eb4-705f-457a-a704-9c03879ebb7c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:26.110304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-522c2eb4-705f-457a-a704-9c03879ebb7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.124785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.126847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.131637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 978/1948] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:26.135958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c2eb4-705f-457a-a704-9c03879ebb7c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.160055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522c2eb4-705f-457a-a704-9c03879ebb7c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.162707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f933b3-1733-4c74-8621-78c602692ef9 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.173523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.174220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.174506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.175286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.175401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '07b67e22df0c41778f0ed0b659bf2da8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '536b7f3bfe3044219f04ddf93aa5f6a6', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.176192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:26.176510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.176918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.177205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.177483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] PATH_INFO: `/v3/projects/3233864d0930452e9148dcaf13b119c1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.178134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.182579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorizing `identity:delete_project(project_id=3233864d0930452e9148dcaf13b119c1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.184570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.210189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 979/1949] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:26.214610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.238403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed2321d-f7ca-450e-9c02-50c566665602 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.245096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.245818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.246149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.246433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.247322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.249558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.264570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3233864d0930452e9148dcaf13b119c1', 'request_id': 'req-d3e87371-0e10-4ad8-8459-e7901f919837'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.270785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3233864d0930452e9148dcaf13b119c1', 'request_id': 'req-d3e87371-0e10-4ad8-8459-e7901f919837'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.279780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 980/1950] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:26 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:26.280928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3233864d0930452e9148dcaf13b119c1', 'request_id': 'req-d3e87371-0e10-4ad8-8459-e7901f919837'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.281068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.293412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] The token cache is being invalidate because project 3233864d0930452e9148dcaf13b119c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:26.293987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.295134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.296977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 971/1951] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:26 2022] DELETE /identity/v3/projects/3233864d0930452e9148dcaf13b119c1 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:26.320679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:26.350172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c719698-5ebf-47a7-a367-3cb06f43ae00 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.355454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.356172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.356465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.356750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.357450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.359380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.379034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 981/1952] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:26 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:26.383220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.384765 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.399769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:26.486164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e87371-0e10-4ad8-8459-e7901f919837 tempest-VolumeQuotaClassesTest-1197326756 tempest-VolumeQuotaClassesTest-1197326756-project-admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.492741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.493770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.494208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.494655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.496043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.498238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.500045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.501856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.520238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 972/1953] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:26.524174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.529571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d071f2-73c0-4313-a14d-4ad0be343cc7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.536507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.536856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac40f2-c97d-469d-a95d-1998a859e172 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.537978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.538546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.539020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] PATH_INFO: `/v3/users/e473ef0d1201403b9627efe2faaba3b9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.540633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e473ef0d1201403b9627efe2faaba3b9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.543134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.543921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.544272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.544486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.544628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.545442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.547786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.564770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 973/1954] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:26.567715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.578623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481fbec-92de-4a17-95e5-11a685195975 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.585387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55c7a1-cf56-41b5-b0f5-38e864d5ea3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.586956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c7a1-cf56-41b5-b0f5-38e864d5ea3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.586956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c7a1-cf56-41b5-b0f5-38e864d5ea3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.587149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55c7a1-cf56-41b5-b0f5-38e864d5ea3d admin admin] PATH_INFO: `/v3/projects/967c5e9eefd54932b56cf15d9ef94caf/users/f171d8f506ea4ad0abd4949d026c795c/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.607242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55c7a1-cf56-41b5-b0f5-38e864d5ea3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=967c5e9eefd54932b56cf15d9ef94caf, user_id=f171d8f506ea4ad0abd4949d026c795c, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.610531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55c7a1-cf56-41b5-b0f5-38e864d5ea3d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.622364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e473ef0d1201403b9627efe2faaba3b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.637738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 974/1955] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:26 2022] PUT /identity/v3/projects/967c5e9eefd54932b56cf15d9ef94caf/users/f171d8f506ea4ad0abd4949d026c795c/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:26.640280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.640640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e473ef0d1201403b9627efe2faaba3b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.640839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.641122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.641311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e473ef0d1201403b9627efe2faaba3b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.656388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e473ef0d1201403b9627efe2faaba3b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.664234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbd306a8-7e79-475b-9855-f8a67f5dcf22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e473ef0d1201403b9627efe2faaba3b9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:26.671774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 982/1956] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:26 2022] DELETE /identity/v3/users/e473ef0d1201403b9627efe2faaba3b9 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:26.672882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] MFA Rules not processed for user `f171d8f506ea4ad0abd4949d026c795c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:26.674213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.676263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.676263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.676524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.695628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:26.696993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.745390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 975/1957] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:26.749755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.751444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.780706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 983/1958] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:26.789590 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.791199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:26.841272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4c455-557d-4432-88f0-121e6cbc98a6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.846221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:26.853271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.854427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.857122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.857600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.858864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.862734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.870711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8cc1c3-7a6d-483d-a148-99919f2e58b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:26.875057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:26.876456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:26.877592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:26.877978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:26.878377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:26.899944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:26.905483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:26.920943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 976/1959] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:26 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:26.922199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:26.930421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 984/1960] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:26.996295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20d4de-3f6b-4292-bf43-a8c997038480 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.001304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.001983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.002281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.002541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.003255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.005231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.019445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.021951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.033028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 977/1961] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:26 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:27.036579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.041859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41437a3a-54d3-4743-bb69-a0b4b6214430 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.046999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f747da7-d3c2-42c6-a7d8-cfaadd5ec294 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.047730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.048841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.049132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.049437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] PATH_INFO: `/v3/users/09e021c1cd054b5eb141621c92973df9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.051389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09e021c1cd054b5eb141621c92973df9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.052850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.053733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.053964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.054065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.054281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.055018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.056769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.069399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 978/1962] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:27.074404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.084691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760c7da-ee2b-467a-9a8c-5400bc4cee68 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.090838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.091659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.091980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.092375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] PATH_INFO: `/v3/projects/1fda236fc52d452e81e866903c837ee2/users/d1baf9ce418746588f216fc48bc7d646/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.111232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fda236fc52d452e81e866903c837ee2, user_id=d1baf9ce418746588f216fc48bc7d646, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.117571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.119668 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09e021c1cd054b5eb141621c92973df9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.128738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09e021c1cd054b5eb141621c92973df9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.129335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09e021c1cd054b5eb141621c92973df9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.133249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 979/1963] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:27 2022] PUT /identity/v3/projects/1fda236fc52d452e81e866903c837ee2/users/d1baf9ce418746588f216fc48bc7d646/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:27.139626 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.141469 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.149800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09e021c1cd054b5eb141621c92973df9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.157949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fb769f6f-a713-4af6-8d09-1862c5263434 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09e021c1cd054b5eb141621c92973df9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.172978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 985/1964] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:27 2022] DELETE /identity/v3/users/09e021c1cd054b5eb141621c92973df9 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:27.176036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.176394 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.177072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.201248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:27.202528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.230417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5efb40-6034-4ee7-b010-8480200b186f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.295923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 986/1965] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:27.301650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:27.313332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.316098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.317026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.317382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.318029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.318637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.320612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.321789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.338162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 980/1966] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:27 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:27.343872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.360743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126abbb-37d6-4b68-95d0-4f1896a3d9cc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.367121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b98cba1-a8aa-405e-b76e-71e4fd6d70ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.368080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98cba1-a8aa-405e-b76e-71e4fd6d70ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.368910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98cba1-a8aa-405e-b76e-71e4fd6d70ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.369592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b98cba1-a8aa-405e-b76e-71e4fd6d70ce admin admin] PATH_INFO: `/v3/projects/1fda236fc52d452e81e866903c837ee2/users/d1baf9ce418746588f216fc48bc7d646/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.372624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98cba1-a8aa-405e-b76e-71e4fd6d70ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=1fda236fc52d452e81e866903c837ee2, user_id=d1baf9ce418746588f216fc48bc7d646, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.376813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b98cba1-a8aa-405e-b76e-71e4fd6d70ce admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.389104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e39f38-c82b-41b1-aed2-ceb47d754d68 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.392521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 981/1967] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:27 2022] PUT /identity/v3/projects/1fda236fc52d452e81e866903c837ee2/users/d1baf9ce418746588f216fc48bc7d646/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:27.393032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:27.411319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.411610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.411909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.453189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] MFA Rules not processed for user `d1baf9ce418746588f216fc48bc7d646`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:27.463900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.468883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.469764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.470514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.471039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.565704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.569630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.595466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 982/1968] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:27 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:27.596960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 987/1969] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:27.606853 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.607220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.608516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.608652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.624377 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc9e91-4125-4a82-8b57-a7088d227d67 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.629257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.630247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.630426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.630787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] PATH_INFO: `/v3/projects/81ce412c59424901bc392bccfa701c18` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.632241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81ce412c59424901bc392bccfa701c18)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.634536 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.698052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81ce412c59424901bc392bccfa701c18', 'request_id': 'req-475ea6a4-6f35-4f55-92d2-dcb522ffa219'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.701674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32aa083c-874b-4267-ae51-ada60c8b4ed3 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.707819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81ce412c59424901bc392bccfa701c18', 'request_id': 'req-475ea6a4-6f35-4f55-92d2-dcb522ffa219'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.719201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81ce412c59424901bc392bccfa701c18', 'request_id': 'req-475ea6a4-6f35-4f55-92d2-dcb522ffa219'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.746512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] The token cache is being invalidate because project 81ce412c59424901bc392bccfa701c18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:27.746814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.748159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.756543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 983/1970] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:27 2022] DELETE /identity/v3/projects/81ce412c59424901bc392bccfa701c18 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:27.758099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:27.761275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.835435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:27.840526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.841245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.841572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.841944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.846870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:27.866811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475ea6a4-6f35-4f55-92d2-dcb522ffa219 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:27.872911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:27.874245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:27.874534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:27.875412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] PATH_INFO: `/v3/users/07b67e22df0c41778f0ed0b659bf2da8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:27.877091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=07b67e22df0c41778f0ed0b659bf2da8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.881179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.906241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:27.955975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:27.957499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:27.971104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07b67e22df0c41778f0ed0b659bf2da8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.978235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 988/1971] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:27.980633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07b67e22df0c41778f0ed0b659bf2da8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.981308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07b67e22df0c41778f0ed0b659bf2da8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:27.989550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07b67e22df0c41778f0ed0b659bf2da8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.003188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07b67e22df0c41778f0ed0b659bf2da8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.011833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 984/1972] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:27 2022] DELETE /identity/v3/users/07b67e22df0c41778f0ed0b659bf2da8 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:28.092334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:28.093771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.098866 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:28.100229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.128639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037af175-fe3a-4d7c-8fa4-da7a521e3ebb service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:28.201271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1e83b5-43d9-4086-8e22-f9ca790bfa3c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:28.208591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:28.209560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.209866 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.210142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] PATH_INFO: `/v3/projects/24e4caaa20604a38854e0c038ece9459` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.211547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] RBAC: Authorizing `identity:delete_project(project_id=24e4caaa20604a38854e0c038ece9459)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:28.219846 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:28.220531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.220827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.221111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.223033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:28.225166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.284600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24e4caaa20604a38854e0c038ece9459', 'request_id': 'req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.293834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24e4caaa20604a38854e0c038ece9459', 'request_id': 'req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.308715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24e4caaa20604a38854e0c038ece9459', 'request_id': 'req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.313475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:28.328088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] The token cache is being invalidate because project 24e4caaa20604a38854e0c038ece9459 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:28.328088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.328785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.331099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 985/1973] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:28 2022] DELETE /identity/v3/projects/24e4caaa20604a38854e0c038ece9459 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:28.334895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:28.336385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.344186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:28.345735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:28.356140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 989/1974] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:28 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:28.431265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6f80f7-e51c-4887-9e90-5e5bbc4b9352 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:28.437690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:28.438728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.439020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.439306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] PATH_INFO: `/v3/users/9d2d57fe427546a8b7aeb6ff4f46e0a8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.440648 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d2d57fe427546a8b7aeb6ff4f46e0a8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:28.442719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:28.453800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:28.455914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.474906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af72bb8e-f2dd-452e-aa98-dc9672678c74 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:28.484120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:28.485320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.485607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.485923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] PATH_INFO: `/v3/projects/536b7f3bfe3044219f04ddf93aa5f6a6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.487215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] RBAC: Authorizing `identity:delete_project(project_id=536b7f3bfe3044219f04ddf93aa5f6a6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:28.492582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:28.517958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d2d57fe427546a8b7aeb6ff4f46e0a8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.528990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d2d57fe427546a8b7aeb6ff4f46e0a8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.529836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d2d57fe427546a8b7aeb6ff4f46e0a8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.540017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d2d57fe427546a8b7aeb6ff4f46e0a8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.559199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1dbf9be9-7f55-43b6-9227-295a011d1cd0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d2d57fe427546a8b7aeb6ff4f46e0a8'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.568389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 986/1975] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:28 2022] DELETE /identity/v3/users/9d2d57fe427546a8b7aeb6ff4f46e0a8 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:28.572780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '536b7f3bfe3044219f04ddf93aa5f6a6', 'request_id': 'req-70ddc029-be4f-4bce-918d-1bd34fc5281a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.582140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '536b7f3bfe3044219f04ddf93aa5f6a6', 'request_id': 'req-70ddc029-be4f-4bce-918d-1bd34fc5281a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.591865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '536b7f3bfe3044219f04ddf93aa5f6a6', 'request_id': 'req-70ddc029-be4f-4bce-918d-1bd34fc5281a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.601126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:28.607086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] The token cache is being invalidate because project 536b7f3bfe3044219f04ddf93aa5f6a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:28.607805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.608918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-70ddc029-be4f-4bce-918d-1bd34fc5281a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:28.618857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b35d7-c9fd-4059-a376-169eb88cf0d1 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.618857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b35d7-c9fd-4059-a376-169eb88cf0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.618857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888b35d7-c9fd-4059-a376-169eb88cf0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.621349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 990/1976] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:28 2022] DELETE /identity/v3/projects/536b7f3bfe3044219f04ddf93aa5f6a6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:28.643438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-888b35d7-c9fd-4059-a376-169eb88cf0d1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:28.645074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-888b35d7-c9fd-4059-a376-169eb88cf0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.760291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 987/1977] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:28.769684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4ce0d-3ea3-40f2-9d1c-3b7050c87d79 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.770159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4ce0d-3ea3-40f2-9d1c-3b7050c87d79 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.770248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed4ce0d-3ea3-40f2-9d1c-3b7050c87d79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.801799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-eed4ce0d-3ea3-40f2-9d1c-3b7050c87d79 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:28.811029 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-eed4ce0d-3ea3-40f2-9d1c-3b7050c87d79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.834873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 991/1978] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:28.838976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b35d7-c9fd-4059-a376-169eb88cf0d1 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:28.851549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888b35d7-c9fd-4059-a376-169eb88cf0d1 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:28.864079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:28.865097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.865382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.865660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.866652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:28.866963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:28.868776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:28.878077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:28.884079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 988/1979] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:26:28.888752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4ce0d-3ea3-40f2-9d1c-3b7050c87d79 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:28.898682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed4ce0d-3ea3-40f2-9d1c-3b7050c87d79 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:28.904447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:28.905422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:28.905422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:28.905701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:28.906400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:28.908743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:28.931338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:28.934410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:28.948120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 992/1980] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:28 2022] POST /identity/v3/projects => generated 393 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:28.950410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab94c6-87b4-4404-972d-0287f4777c5c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:28.952500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.014704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.015679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.015998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.016278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] PATH_INFO: `/v3/projects/20af2c18f0ff4a7a8a70290cc1c64f2e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.017487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] RBAC: Authorizing `identity:delete_project(project_id=20af2c18f0ff4a7a8a70290cc1c64f2e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.019403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.020698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffa3877-f1d2-4583-bd0a-f389c5559cd0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.025545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.026219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.026621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.026908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.027653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.029617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.052784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 993/1981] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:28 2022] POST /identity/v3/users => generated 297 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:29.053877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.064014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e6147a-eb3d-404a-8a11-f01b4778691b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.070365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20af2c18f0ff4a7a8a70290cc1c64f2e', 'request_id': 'req-ff5ef969-1955-444f-987a-b354cf5c308c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:29.077220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20af2c18f0ff4a7a8a70290cc1c64f2e', 'request_id': 'req-ff5ef969-1955-444f-987a-b354cf5c308c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:29.084995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20af2c18f0ff4a7a8a70290cc1c64f2e', 'request_id': 'req-ff5ef969-1955-444f-987a-b354cf5c308c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:29.095821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] The token cache is being invalidate because project 20af2c18f0ff4a7a8a70290cc1c64f2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:29.096162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:29.096812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ff5ef969-1955-444f-987a-b354cf5c308c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:29.098427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 989/1982] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:28 2022] DELETE /identity/v3/projects/20af2c18f0ff4a7a8a70290cc1c64f2e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:29.102940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc90b-ee45-4c1b-a5bb-a4aa82dfd61e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.103482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc90b-ee45-4c1b-a5bb-a4aa82dfd61e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.103482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfcc90b-ee45-4c1b-a5bb-a4aa82dfd61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.122153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bdfcc90b-ee45-4c1b-a5bb-a4aa82dfd61e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:29.123289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bdfcc90b-ee45-4c1b-a5bb-a4aa82dfd61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:29.124837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.125520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.125811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.126094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.126829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.128476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.131229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 990/1983] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:29.139259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1e99e-8427-4af4-be09-344898dd5977 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.139551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1e99e-8427-4af4-be09-344898dd5977 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.139868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1e99e-8427-4af4-be09-344898dd5977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.143850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 994/1984] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:29.148222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.149886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:29.157197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d2a1e99e-8427-4af4-be09-344898dd5977 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:29.158370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2a1e99e-8427-4af4-be09-344898dd5977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:29.162026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d7c7c-56f0-48d7-9598-285cd367f306 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.166312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 991/1985] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:29.166613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.167323 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.167608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.167922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.168669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.170319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1e99e-8427-4af4-be09-344898dd5977 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.170447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.179055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1e99e-8427-4af4-be09-344898dd5977 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.183714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 995/1986] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:29.184299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.184936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.185215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.185492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.186197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:29.186504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.188111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.188748 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.199042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf950ae-35a5-4b3b-b5b3-022bb899d2be admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.203712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 992/1987] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:26:29.203849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d10ace-6638-45ba-af2a-f96eab876752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.204625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d10ace-6638-45ba-af2a-f96eab876752 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.204906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d10ace-6638-45ba-af2a-f96eab876752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.205181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d10ace-6638-45ba-af2a-f96eab876752 admin admin] PATH_INFO: `/v3/projects/de5aa67a5be843d4a82687f850a55ca1/users/5260002f61cf49308fefa7f99c3c7fe0/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.206682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.216014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee7070-0b47-4d16-8ad9-01f54e6bd164 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.220045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d10ace-6638-45ba-af2a-f96eab876752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de5aa67a5be843d4a82687f850a55ca1, user_id=5260002f61cf49308fefa7f99c3c7fe0, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.220654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.221273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.221552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.221821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.223508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.223636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d10ace-6638-45ba-af2a-f96eab876752 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.229344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.238064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 996/1988] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:29 2022] PUT /identity/v3/projects/de5aa67a5be843d4a82687f850a55ca1/users/5260002f61cf49308fefa7f99c3c7fe0/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:29.249273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf439ba-5b06-405e-9886-bfbfd461e844 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.249643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf439ba-5b06-405e-9886-bfbfd461e844 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.250026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf439ba-5b06-405e-9886-bfbfd461e844 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.265871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 993/1989] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/projects => generated 377 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:29.270590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.284038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1bf439ba-5b06-405e-9886-bfbfd461e844 None None] MFA Rules not processed for user `5260002f61cf49308fefa7f99c3c7fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:29.285480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1bf439ba-5b06-405e-9886-bfbfd461e844 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:29.296697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:29.326103 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:29.357576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a8bd6-d117-4fae-be6c-9de1aa13a895 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.363512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.364222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.364566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.364854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.365551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.367501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.372822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 997/1990] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:29.377646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf439ba-5b06-405e-9886-bfbfd461e844 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.389569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf439ba-5b06-405e-9886-bfbfd461e844 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.392321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 994/1991] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/users => generated 289 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:29.396219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.398560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.399275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.399562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.399902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.400670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.402649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.406970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e53854-3e33-400e-b63b-34529021cadc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.412110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.412780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.413067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.413344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.414061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.415659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.427877 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 995/1992] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:29.431952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.437758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 998/1993] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:29.438626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.443310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d24054-59b4-4bef-8680-dd2b73eaf399 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.521413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc541ef-ac0d-4717-aa3f-e76609339ca2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.526993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.527676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.527996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.528279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.528980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.531019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.532184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.532558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.532850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.533214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.535902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.535902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.547304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 996/1994] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:29.551453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.557582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 999/1995] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/users => generated 302 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:29.562135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.564189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5f6f7d-e935-4e35-8b44-4403b15f57e3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.572623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e229014-b59c-4cda-85c9-e354ab33a195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.573562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e229014-b59c-4cda-85c9-e354ab33a195 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.573986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e229014-b59c-4cda-85c9-e354ab33a195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.574296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e229014-b59c-4cda-85c9-e354ab33a195 admin admin] PATH_INFO: `/v3/projects/1bba346cb691427bb207d0bd20e5007f/users/2222b50b6ad94af78d98860deed08560/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.575308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89783e5-643b-4936-8533-34162fca5ea4 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.583392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.584142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.584430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.584715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.585453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.587076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.592205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e229014-b59c-4cda-85c9-e354ab33a195 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bba346cb691427bb207d0bd20e5007f, user_id=2222b50b6ad94af78d98860deed08560, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.594875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e229014-b59c-4cda-85c9-e354ab33a195 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.598085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1000/1996] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:29.602104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.610175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 997/1997] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:29 2022] PUT /identity/v3/projects/1bba346cb691427bb207d0bd20e5007f/users/2222b50b6ad94af78d98860deed08560/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:29.614397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4640bc37-1de6-4e58-b4f5-4bc578d04fc8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.621858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2be34-447e-45b0-85f8-f8ab2a9bbe21 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.622352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2be34-447e-45b0-85f8-f8ab2a9bbe21 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.622651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2be34-447e-45b0-85f8-f8ab2a9bbe21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.655192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-26d2be34-447e-45b0-85f8-f8ab2a9bbe21 None None] MFA Rules not processed for user `2222b50b6ad94af78d98860deed08560`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:29.659301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-26d2be34-447e-45b0-85f8-f8ab2a9bbe21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:29.677885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.679378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.679693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.680005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] PATH_INFO: `/v3/projects/6f20a3088a0f42c9af359dc131ef2b7f/users/099101ec72eb4ef3b6287c514f0987ef/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.694941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:29.698111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f20a3088a0f42c9af359dc131ef2b7f, user_id=099101ec72eb4ef3b6287c514f0987ef, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.701143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.720042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1001/1998] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:29 2022] PUT /identity/v3/projects/6f20a3088a0f42c9af359dc131ef2b7f/users/099101ec72eb4ef3b6287c514f0987ef/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:29.724082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.742752 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 998/1999] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:29.747304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2be34-447e-45b0-85f8-f8ab2a9bbe21 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:29.758504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2be34-447e-45b0-85f8-f8ab2a9bbe21 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:29.779250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:29.973882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:29.974685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:29.974968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:29.975333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:29.976058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:29.978242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:29.994002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615d8bd-c1b6-4699-a542-0ca04ae6946a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.013702 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 999/2000] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:29 2022] POST /identity/v3/projects => generated 379 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:30.016890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.017605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.017901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.018267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.018940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.023965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.024302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.038670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1002/2001] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:30.041672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.064210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f55b069-b41e-41cd-80a1-44fec18fc1d7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.170448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717d55b-6f3a-41be-b350-174742488f1c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.185815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.186526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.186813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.187094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.187832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.189859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.210152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.210364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.210814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.211704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] PATH_INFO: `/v3/projects/6f20a3088a0f42c9af359dc131ef2b7f/users/099101ec72eb4ef3b6287c514f0987ef/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.217094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f20a3088a0f42c9af359dc131ef2b7f, user_id=099101ec72eb4ef3b6287c514f0987ef, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.225168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.226714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1000/2002] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:30 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:30.227669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.229541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:30.238674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1003/2003] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:30 2022] PUT /identity/v3/projects/6f20a3088a0f42c9af359dc131ef2b7f/users/099101ec72eb4ef3b6287c514f0987ef/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:30.239620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.325483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702911f-b834-42a8-b975-44903ba3fc72 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.349085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb23b47-729d-42ef-9750-067f47debe97 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.355987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb76cd09-2dbc-408c-8a48-9259788478ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.357266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76cd09-2dbc-408c-8a48-9259788478ec admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.357919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76cd09-2dbc-408c-8a48-9259788478ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.358405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb76cd09-2dbc-408c-8a48-9259788478ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.359497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76cd09-2dbc-408c-8a48-9259788478ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.363086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb76cd09-2dbc-408c-8a48-9259788478ec admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.392771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1004/2004] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:30.394436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e6f4e-8fda-4df7-823b-0463edd63376 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.394740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e6f4e-8fda-4df7-823b-0463edd63376 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.395651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e6f4e-8fda-4df7-823b-0463edd63376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.396512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.397177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.397594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.399673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.404814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:30.410883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:30.434312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-545e6f4e-8fda-4df7-823b-0463edd63376 None None] MFA Rules not processed for user `099101ec72eb4ef3b6287c514f0987ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:30.435936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-545e6f4e-8fda-4df7-823b-0463edd63376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:30.441769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:30.491566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.493139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.503606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1001/2005] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:30 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:30.504476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.522754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6892e200-917e-4e2e-a4d3-891261b56d8f service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.528402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.529458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.530027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.530183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.531002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.532776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.546094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1002/2006] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:30.546983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.548747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:30.557237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1005/2007] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:30 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:30.558764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e6f4e-8fda-4df7-823b-0463edd63376 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.571621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1e669a-e223-4a2f-b6e0-9fd130b0c080 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.572096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e6f4e-8fda-4df7-823b-0463edd63376 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.577070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.577837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00dfcc4-5258-4cbf-be70-0ba2d2997649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.578064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.578493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.578741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfcc4-5258-4cbf-be70-0ba2d2997649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.579031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfcc4-5258-4cbf-be70-0ba2d2997649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.579231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.579348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00dfcc4-5258-4cbf-be70-0ba2d2997649 admin admin] PATH_INFO: `/v3/projects/a12c5d41c3ee49ddbee93cd8d7c3cc15/users/17fd0a0d0a9f4b61a57091684b86b80c/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.582378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:30.602272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dfcc4-5258-4cbf-be70-0ba2d2997649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a12c5d41c3ee49ddbee93cd8d7c3cc15, user_id=17fd0a0d0a9f4b61a57091684b86b80c, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.606563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00dfcc4-5258-4cbf-be70-0ba2d2997649 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.627417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1006/2008] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:30 2022] PUT /identity/v3/projects/a12c5d41c3ee49ddbee93cd8d7c3cc15/users/17fd0a0d0a9f4b61a57091684b86b80c/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:30.638692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44048f33-7244-423a-84bd-398b5f27f176 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.638892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44048f33-7244-423a-84bd-398b5f27f176 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.639124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44048f33-7244-423a-84bd-398b5f27f176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.672030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.673400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.678343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-44048f33-7244-423a-84bd-398b5f27f176 None None] MFA Rules not processed for user `17fd0a0d0a9f4b61a57091684b86b80c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:30.680433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-44048f33-7244-423a-84bd-398b5f27f176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:30.684288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1003/2009] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:30 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:30.757107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1007/2010] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:30.761344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44048f33-7244-423a-84bd-398b5f27f176 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.761457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.771212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db43436d-c5a7-48c3-8db8-43cffc1d4972 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.847576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44048f33-7244-423a-84bd-398b5f27f176 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.853023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.853893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.854258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.855260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.856075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.858728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.859395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.859680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.860021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.862203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.881437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:30.899833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1008/2011] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:30 2022] POST /identity/v3/projects => generated 379 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:30.903366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:30.928886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:30.968213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.970146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:30.982361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1004/2012] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:30 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:30.983187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb04ff61-8905-48a2-ac4f-49a0703aef49 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:30.989097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:30.990104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:30.990504 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:30.991034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:30.991791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:30.993884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.020091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1009/2013] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:30 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:31.026118 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.033637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970a6f6f-c06b-4603-9f79-aea4634985ed service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.035237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.038859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.039846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.040121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.040399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.041242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.043533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.054301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1005/2014] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:31.059184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.070365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc5ea7-0c3c-4883-819e-fbe283bae6ba admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.078321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.079326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.079642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.079952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] PATH_INFO: `/v3/projects/bde99b3a46cc4e059ee1965f24e1f30d/users/e294bffa880c422ea82bc33fd75f46b7/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.096583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde99b3a46cc4e059ee1965f24e1f30d, user_id=e294bffa880c422ea82bc33fd75f46b7, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.099573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.099911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a92723e-3c0e-48b1-ba72-a5d087c5005b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.105215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.105957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.106272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.106597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.114952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1006/2015] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:31 2022] PUT /identity/v3/projects/bde99b3a46cc4e059ee1965f24e1f30d/users/e294bffa880c422ea82bc33fd75f46b7/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:31.118802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.178508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.180262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.190855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1010/2016] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:31.249901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716ed72a-7a92-462f-a0a4-18f7ec595b03 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.255635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa096c2-5690-4cd5-acf2-605e7ae97842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.256476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa096c2-5690-4cd5-acf2-605e7ae97842 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.256812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa096c2-5690-4cd5-acf2-605e7ae97842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.257141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa096c2-5690-4cd5-acf2-605e7ae97842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.257913 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa096c2-5690-4cd5-acf2-605e7ae97842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.259601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa096c2-5690-4cd5-acf2-605e7ae97842 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.271800 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1007/2017] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:31.275528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.288011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa2ce4-9b22-49da-a20f-901f6e30ee23 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.293474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.296316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.296613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.296892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] PATH_INFO: `/v3/projects/bde99b3a46cc4e059ee1965f24e1f30d/users/e294bffa880c422ea82bc33fd75f46b7/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.298934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bde99b3a46cc4e059ee1965f24e1f30d, user_id=e294bffa880c422ea82bc33fd75f46b7, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.304023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.317985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1011/2018] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:31 2022] PUT /identity/v3/projects/bde99b3a46cc4e059ee1965f24e1f30d/users/e294bffa880c422ea82bc33fd75f46b7/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:31.329183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb61f63-5beb-4f2d-a0e7-c4310fdd1a6b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.329566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb61f63-5beb-4f2d-a0e7-c4310fdd1a6b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.329918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb61f63-5beb-4f2d-a0e7-c4310fdd1a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.362064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7cb61f63-5beb-4f2d-a0e7-c4310fdd1a6b None None] MFA Rules not processed for user `e294bffa880c422ea82bc33fd75f46b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:31.363373 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7cb61f63-5beb-4f2d-a0e7-c4310fdd1a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:31.398687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.400620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:31.435107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:31.443531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1008/2019] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:31.455269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb61f63-5beb-4f2d-a0e7-c4310fdd1a6b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.482784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:31.499546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43e9f87-5887-44d9-af54-94027227752e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.519112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.520050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.520328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.520605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] PATH_INFO: `/v3/users/f171d8f506ea4ad0abd4949d026c795c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.522210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f171d8f506ea4ad0abd4949d026c795c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.525836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.539215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb61f63-5beb-4f2d-a0e7-c4310fdd1a6b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.548008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.548687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.555876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.555876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.584611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.586621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.599091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1009/2020] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:31.610370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f171d8f506ea4ad0abd4949d026c795c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:31.620152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f171d8f506ea4ad0abd4949d026c795c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:31.624165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f171d8f506ea4ad0abd4949d026c795c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:31.645795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f171d8f506ea4ad0abd4949d026c795c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:31.646836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.653649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f312c2b4-750b-4829-a011-13c669e0726d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f171d8f506ea4ad0abd4949d026c795c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:31.656945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29450ce-b529-4a9e-a0ff-d43679039e08 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.662566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1012/2021] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:31 2022] DELETE /identity/v3/users/f171d8f506ea4ad0abd4949d026c795c => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:31.663546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.664060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.664656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.664934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.665038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.665423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.665827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.667971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:31.687616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:31.687616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:31.735990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.737470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.746949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1010/2022] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:31.753114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1013/2023] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:31.763055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:31.765077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:31.844275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6883e53-d1fb-4624-8707-8294d926ce20 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:31.895711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:31.896522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:31.896903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:31.897374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:31.908490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:31.910189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:31.923645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1011/2024] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:32.264891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:32.266417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:32.279683 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6fa2c3-0614-4a65-b636-ffc92250b495 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:32.284415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:32.285412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:32.285761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:32.286098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] PATH_INFO: `/v3/projects/967c5e9eefd54932b56cf15d9ef94caf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:32.287336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=967c5e9eefd54932b56cf15d9ef94caf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:32.290532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:32.334420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:32.343404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '967c5e9eefd54932b56cf15d9ef94caf', 'request_id': 'req-73359577-2382-4c5d-ae4e-21b6710f4ac7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.350606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '967c5e9eefd54932b56cf15d9ef94caf', 'request_id': 'req-73359577-2382-4c5d-ae4e-21b6710f4ac7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.357315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '967c5e9eefd54932b56cf15d9ef94caf', 'request_id': 'req-73359577-2382-4c5d-ae4e-21b6710f4ac7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.371913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] The token cache is being invalidate because project 967c5e9eefd54932b56cf15d9ef94caf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:32.372211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.373042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-73359577-2382-4c5d-ae4e-21b6710f4ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.375069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1014/2025] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:32 2022] DELETE /identity/v3/projects/967c5e9eefd54932b56cf15d9ef94caf => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:32.379022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:32.388847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586f8c0f-2cc3-40ef-9b98-895ad74eca3b service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:32.404989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:32.464503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:32.465507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:32.465912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:32.466288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] PATH_INFO: `/v3/users/d1baf9ce418746588f216fc48bc7d646` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:32.467609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1baf9ce418746588f216fc48bc7d646)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:32.469681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:32.535854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1baf9ce418746588f216fc48bc7d646'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.546234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1baf9ce418746588f216fc48bc7d646'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.546928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1baf9ce418746588f216fc48bc7d646'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.556120 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1baf9ce418746588f216fc48bc7d646'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.571559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d11442a9-e735-45c5-af8d-1211b5561c70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1baf9ce418746588f216fc48bc7d646'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:32.578616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1012/2026] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:32 2022] DELETE /identity/v3/users/d1baf9ce418746588f216fc48bc7d646 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:32.635005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52498693-5629-4188-8e86-48cb20b9a813 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:32.635620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52498693-5629-4188-8e86-48cb20b9a813 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:32.636077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52498693-5629-4188-8e86-48cb20b9a813 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:32.637750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1015/2027] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:26:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:26:32.646118 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:32.646524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:32.646950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:32.668320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:32.669642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:32.765987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1013/2028] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:26:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:32.778606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52498693-5629-4188-8e86-48cb20b9a813 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:32.780433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-52498693-5629-4188-8e86-48cb20b9a813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:32.798900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52498693-5629-4188-8e86-48cb20b9a813 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:32.804615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94790c4-9be2-4173-9774-af3b00b75dbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:32.805365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94790c4-9be2-4173-9774-af3b00b75dbc service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:32.805723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94790c4-9be2-4173-9774-af3b00b75dbc service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:32.806075 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94790c4-9be2-4173-9774-af3b00b75dbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:32.815702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94790c4-9be2-4173-9774-af3b00b75dbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:32.817339 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94790c4-9be2-4173-9774-af3b00b75dbc service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:32.828384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1016/2029] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:32.945001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:32.946608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:32.990941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:33.024060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8438b822-e404-4db5-82c6-e302bcd07a20 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.029495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.030389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.030666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.030940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] PATH_INFO: `/v3/projects/1fda236fc52d452e81e866903c837ee2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.032307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1fda236fc52d452e81e866903c837ee2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.034243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.079788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1fda236fc52d452e81e866903c837ee2', 'request_id': 'req-a3f253e9-ef90-469b-90bf-342df4c74728'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:33.085687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1fda236fc52d452e81e866903c837ee2', 'request_id': 'req-a3f253e9-ef90-469b-90bf-342df4c74728'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:33.093477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1fda236fc52d452e81e866903c837ee2', 'request_id': 'req-a3f253e9-ef90-469b-90bf-342df4c74728'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:33.103907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] The token cache is being invalidate because project 1fda236fc52d452e81e866903c837ee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:33.104193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:33.104825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a3f253e9-ef90-469b-90bf-342df4c74728 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:33.106328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1014/2030] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:32 2022] DELETE /identity/v3/projects/1fda236fc52d452e81e866903c837ee2 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:33.111155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255323d8-1a1f-452e-b189-4335168408e9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.111463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255323d8-1a1f-452e-b189-4335168408e9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.111720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255323d8-1a1f-452e-b189-4335168408e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.129408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-255323d8-1a1f-452e-b189-4335168408e9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:33.130608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-255323d8-1a1f-452e-b189-4335168408e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:33.189669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1017/2031] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:33.197465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8b7f-151c-4fb5-9c9f-0e5cdd2de32d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.197757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8b7f-151c-4fb5-9c9f-0e5cdd2de32d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.198036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8b7f-151c-4fb5-9c9f-0e5cdd2de32d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.217419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-28ce8b7f-151c-4fb5-9c9f-0e5cdd2de32d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:33.218628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-28ce8b7f-151c-4fb5-9c9f-0e5cdd2de32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:33.227814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1015/2032] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:33.231728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255323d8-1a1f-452e-b189-4335168408e9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.241328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255323d8-1a1f-452e-b189-4335168408e9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.245943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.246598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.246880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.247156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.247925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:33.248234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.249814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.262206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1018/2033] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:33.266535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8b7f-151c-4fb5-9c9f-0e5cdd2de32d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.275824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ce8b7f-151c-4fb5-9c9f-0e5cdd2de32d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.281069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.281717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.281998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.282271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.282951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.284859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.309982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1016/2034] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:33.314098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.324810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4dd4b6-c36f-44c9-9284-5c8dd721fdc6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.379349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.380052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.380336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.380611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.381294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.383185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.403156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1019/2035] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:33.407413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.417794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04d4a20-85d0-4d48-8b8c-f42d3b19bcfe admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.422388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.423049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.423333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.423611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.424374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.425966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.436714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1017/2036] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:33.440662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.451524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4e6f19-032e-416c-8e52-d72919a0b15d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.456385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c7eb89-67d3-4e86-9378-ae7640418358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.457053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7eb89-67d3-4e86-9378-ae7640418358 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.457341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7eb89-67d3-4e86-9378-ae7640418358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.457624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c7eb89-67d3-4e86-9378-ae7640418358 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.458356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7eb89-67d3-4e86-9378-ae7640418358 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.460020 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c7eb89-67d3-4e86-9378-ae7640418358 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.469904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1020/2037] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:33.473749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.483573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaf653f-43fb-45fd-ae72-4a7b69362c7f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.488218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.489090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.489366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.489637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] PATH_INFO: `/v3/projects/1dc6117c79124a1aab8844d5c1dc7e0d/users/ea6bde782a1447daa0b6d7b6960162ce/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.504302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc6117c79124a1aab8844d5c1dc7e0d, user_id=ea6bde782a1447daa0b6d7b6960162ce, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.506951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.521349 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1018/2038] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:33 2022] PUT /identity/v3/projects/1dc6117c79124a1aab8844d5c1dc7e0d/users/ea6bde782a1447daa0b6d7b6960162ce/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:33.532340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4e0be-bbbf-4c2f-9580-40535fca970e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.532640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4e0be-bbbf-4c2f-9580-40535fca970e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.532926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b4e0be-bbbf-4c2f-9580-40535fca970e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.560547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b5b4e0be-bbbf-4c2f-9580-40535fca970e None None] MFA Rules not processed for user `ea6bde782a1447daa0b6d7b6960162ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:33.561763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b5b4e0be-bbbf-4c2f-9580-40535fca970e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:33.570878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:33.643964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1021/2039] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:33.648241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.709776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:33.714691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d774cc-f155-49b4-ab45-c847ba34915c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.721105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.721941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.722443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.722769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.723670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.725880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.750556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1019/2040] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:33.754612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4e0be-bbbf-4c2f-9580-40535fca970e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.765850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b4e0be-bbbf-4c2f-9580-40535fca970e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.821556 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.822233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.822523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.822802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.824065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.826016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.845530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1022/2041] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:33.849398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.860168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dae441b-ef85-48ed-a9b5-73061dd758e0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.864698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.865343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.865621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.865893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.866605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.868224 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.878925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1020/2042] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:33.882627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.893038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c300a-2e27-454c-9c8d-76ff4396941a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.897647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c61cbb62-d360-419b-b866-55d82cac8b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.898295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbb62-d360-419b-b866-55d82cac8b1a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.898581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbb62-d360-419b-b866-55d82cac8b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.898862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c61cbb62-d360-419b-b866-55d82cac8b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.899582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cbb62-d360-419b-b866-55d82cac8b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.901203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c61cbb62-d360-419b-b866-55d82cac8b1a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.911685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1023/2043] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:33.915431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:33.925315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d47a560-936d-4ce7-be75-68b258d9b072 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:33.929892 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:33.930769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.931047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.931316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] PATH_INFO: `/v3/projects/80cb94d89e4241e091bb5bbd4aaea6c3/users/97962bffb29c4bebb8b6e4c2f7383018/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:33.946914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] RBAC: Authorizing `identity:create_grant(project_id=80cb94d89e4241e091bb5bbd4aaea6c3, user_id=97962bffb29c4bebb8b6e4c2f7383018, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:33.950090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:33.964584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1021/2044] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:33 2022] PUT /identity/v3/projects/80cb94d89e4241e091bb5bbd4aaea6c3/users/97962bffb29c4bebb8b6e4c2f7383018/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:33.975732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9eddfa-2444-4e34-9656-0572de731e21 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:33.975991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9eddfa-2444-4e34-9656-0572de731e21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:33.976271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9eddfa-2444-4e34-9656-0572de731e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.002801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ab9eddfa-2444-4e34-9656-0572de731e21 None None] MFA Rules not processed for user `97962bffb29c4bebb8b6e4c2f7383018`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:34.004178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ab9eddfa-2444-4e34-9656-0572de731e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:34.075793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1024/2045] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:34.084004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:34.146433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73082f-40ce-49cd-b410-b22007bd693f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:34.151099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:34.151811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.152097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.152374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.153061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:34.154970 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:34.183911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1022/2046] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:34 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:34.186012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9eddfa-2444-4e34-9656-0572de731e21 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:34.197190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9eddfa-2444-4e34-9656-0572de731e21 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:34.210959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:34.264272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:34.265220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.267881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.267881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.267881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:34.269537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:34.300361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1025/2047] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:34 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:34.302625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:34.315308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91d02a-3d4b-4f02-a8fb-88e9f477055e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:34.320783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:34.321453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.321739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.322440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.322741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:34.324930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:34.337591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1023/2048] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:34.341681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:34.353999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874865a8-16b6-4f34-80e8-14708722407b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:34.359016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:34.359954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.360240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.360519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] PATH_INFO: `/v3/projects/64773073271c4ef9b6fd893d4505175b/users/8c2326faea9d4a569089cb5dbbf558ce/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.377458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64773073271c4ef9b6fd893d4505175b, user_id=8c2326faea9d4a569089cb5dbbf558ce, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:34.380248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:34.393697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1026/2049] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:34 2022] PUT /identity/v3/projects/64773073271c4ef9b6fd893d4505175b/users/8c2326faea9d4a569089cb5dbbf558ce/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:34.397755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:34.408606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a70fc-d99b-4a7f-8642-eb8176b987e6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:34.470854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824d7f06-fb09-4db5-97de-d37e99d02f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:34.471625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7f06-fb09-4db5-97de-d37e99d02f39 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.471969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7f06-fb09-4db5-97de-d37e99d02f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.472276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824d7f06-fb09-4db5-97de-d37e99d02f39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.472997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d7f06-fb09-4db5-97de-d37e99d02f39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:34.474582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824d7f06-fb09-4db5-97de-d37e99d02f39 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:34.488665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1024/2050] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:34.489782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:34.500941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb37cf-0bf7-4bbf-91be-d410ab12724d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:34.506333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:34.507303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.507674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.508047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] PATH_INFO: `/v3/projects/64773073271c4ef9b6fd893d4505175b/users/8c2326faea9d4a569089cb5dbbf558ce/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.510272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64773073271c4ef9b6fd893d4505175b, user_id=8c2326faea9d4a569089cb5dbbf558ce, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:34.513082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:34.524981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1027/2051] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:34 2022] PUT /identity/v3/projects/64773073271c4ef9b6fd893d4505175b/users/8c2326faea9d4a569089cb5dbbf558ce/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:34.535101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e96b-71c6-4337-a424-bc3fc38c001f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.535388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e96b-71c6-4337-a424-bc3fc38c001f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.535667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530e96b-71c6-4337-a424-bc3fc38c001f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.562760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5530e96b-71c6-4337-a424-bc3fc38c001f None None] MFA Rules not processed for user `8c2326faea9d4a569089cb5dbbf558ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:34.564101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5530e96b-71c6-4337-a424-bc3fc38c001f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:34.569321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:34.633827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1025/2052] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:34.645561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:34.647251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:34.714564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2865931b-4ee5-4a1b-ba3a-d8a98b5d5d52 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:34.723126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:34.724688 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:34.725628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:34.726305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:34.829935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:34.832844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:34.850261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1028/2053] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:36.535989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e96b-71c6-4337-a424-bc3fc38c001f None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:36.551388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530e96b-71c6-4337-a424-bc3fc38c001f None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:36.564845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:36.565805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:36.566097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:36.566372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:36.635519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:36.637169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:36.646605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1026/2054] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:36 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:37.275374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:37.277258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.297758 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:37.299343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.321624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:37.386711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3142413-1350-4fb2-857d-e1a2cfc6275d service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:37.395984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:37.396872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.397156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.397463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] PATH_INFO: `/v3/users/5260002f61cf49308fefa7f99c3c7fe0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.398673 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=5260002f61cf49308fefa7f99c3c7fe0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:37.400639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:37.406181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf99865-ebd3-425b-9b7f-a9d73d699291 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:37.410808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:37.411673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.411975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.412246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] PATH_INFO: `/v3/users/2222b50b6ad94af78d98860deed08560` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.413428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2222b50b6ad94af78d98860deed08560)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:37.415437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:37.449552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5260002f61cf49308fefa7f99c3c7fe0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.457217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5260002f61cf49308fefa7f99c3c7fe0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.457978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5260002f61cf49308fefa7f99c3c7fe0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.462503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:37.465283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2222b50b6ad94af78d98860deed08560'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.470978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5260002f61cf49308fefa7f99c3c7fe0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.475305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2222b50b6ad94af78d98860deed08560'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.476147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2222b50b6ad94af78d98860deed08560'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.478669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5260002f61cf49308fefa7f99c3c7fe0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.483703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2222b50b6ad94af78d98860deed08560'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.485457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1029/2055] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:37 2022] DELETE /identity/v3/users/5260002f61cf49308fefa7f99c3c7fe0 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:37.488532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:37.488532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.497203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4252d4db-57f1-4c7e-9bfd-78d9868d3da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2222b50b6ad94af78d98860deed08560'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.506522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1027/2056] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:37 2022] DELETE /identity/v3/users/2222b50b6ad94af78d98860deed08560 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:37.506522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2baac-a04f-47ac-b2d2-6f0c1f48066d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.506522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2baac-a04f-47ac-b2d2-6f0c1f48066d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.506522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f2baac-a04f-47ac-b2d2-6f0c1f48066d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.523315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-46f2baac-a04f-47ac-b2d2-6f0c1f48066d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:37.524569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46f2baac-a04f-47ac-b2d2-6f0c1f48066d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.549337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858a3f30-1edb-4740-98a0-919483a532ab admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:37.592172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1028/2057] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:37.593861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.594153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.594435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.605280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:37.605874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.606160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.606437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.608802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.614177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:37.615587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.624610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:37.626730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1029/2058] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:37.630286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:37.631498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.692655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:37.694114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:37.706326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbda0fa-c1f7-46b2-b6dc-a1cdc54cb2cb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:37.710233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:37.710969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.711254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.711533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.715187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1030/2059] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:37.716246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:37.717686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.725029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:37.726683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:37.737845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1030/2060] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:37.743776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5374c5ba-698c-42dd-818d-b38346a214b3 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:37.748904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2818ff66-61e3-4ec3-97ab-752af89a3b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:37.749764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ff66-61e3-4ec3-97ab-752af89a3b28 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.750269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ff66-61e3-4ec3-97ab-752af89a3b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.750619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2818ff66-61e3-4ec3-97ab-752af89a3b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.764164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818ff66-61e3-4ec3-97ab-752af89a3b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:37.765956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2818ff66-61e3-4ec3-97ab-752af89a3b28 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:37.778649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1031/2061] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:37.843911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:37.845395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:37.861563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f580ada-133d-4f82-b24c-4368f1a05955 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:37.867234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:37.868391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:37.869177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:37.869177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] PATH_INFO: `/v3/users/c4a986882f1d4d4b8d64cb828ee65616` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:37.871473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4a986882f1d4d4b8d64cb828ee65616)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:37.873759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:37.976113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4a986882f1d4d4b8d64cb828ee65616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.981467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:37.986881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4a986882f1d4d4b8d64cb828ee65616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:37.987598 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4a986882f1d4d4b8d64cb828ee65616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.002837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4a986882f1d4d4b8d64cb828ee65616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.021156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4a986882f1d4d4b8d64cb828ee65616'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.030252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1031/2062] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:37 2022] DELETE /identity/v3/users/c4a986882f1d4d4b8d64cb828ee65616 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:38.036851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:38.037076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:38.037767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:38.067090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:38.068387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:38.156231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1032/2063] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:38.165028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:38.166515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:38.242619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643d702f-9ea2-4428-b57b-00f64cfbeb86 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:38.247798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:38.248647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:38.249077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:38.249674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:38.271894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:38.275860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:38.288527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1032/2064] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:38.289591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:38.291376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:38.323870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0028bf49-86cc-4a46-8024-f2cc7371b990 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:38.341901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:38.342834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:38.343096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:38.343378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] PATH_INFO: `/v3/projects/1bba346cb691427bb207d0bd20e5007f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:38.346651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bba346cb691427bb207d0bd20e5007f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:38.348635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:38.406460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bba346cb691427bb207d0bd20e5007f', 'request_id': 'req-ea25a8a0-13d3-429a-a567-fbdffe02a74a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.421585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bba346cb691427bb207d0bd20e5007f', 'request_id': 'req-ea25a8a0-13d3-429a-a567-fbdffe02a74a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.428658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bba346cb691427bb207d0bd20e5007f', 'request_id': 'req-ea25a8a0-13d3-429a-a567-fbdffe02a74a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.441999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] The token cache is being invalidate because project 1bba346cb691427bb207d0bd20e5007f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:38.442574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.443309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.446382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1033/2065] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:38 2022] DELETE /identity/v3/projects/1bba346cb691427bb207d0bd20e5007f => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:38.450121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:38.474868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a12b78f-4aec-49e4-81a0-3b2178a8b018 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:38.490823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:38.496297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:38.546175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:38.580983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:38.582894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:38.583281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:38.583665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] PATH_INFO: `/v3/users/17fd0a0d0a9f4b61a57091684b86b80c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:38.586488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17fd0a0d0a9f4b61a57091684b86b80c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:38.590258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:38.610010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25a8a0-13d3-429a-a567-fbdffe02a74a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:38.615423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:38.616637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:38.617079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:38.617514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] PATH_INFO: `/v3/projects/de5aa67a5be843d4a82687f850a55ca1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:38.619095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=de5aa67a5be843d4a82687f850a55ca1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:38.622207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:38.663500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17fd0a0d0a9f4b61a57091684b86b80c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.674061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17fd0a0d0a9f4b61a57091684b86b80c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.675859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17fd0a0d0a9f4b61a57091684b86b80c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.684759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17fd0a0d0a9f4b61a57091684b86b80c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.689100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de5aa67a5be843d4a82687f850a55ca1', 'request_id': 'req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.697167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:38.697586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de5aa67a5be843d4a82687f850a55ca1', 'request_id': 'req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.704530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17fd0a0d0a9f4b61a57091684b86b80c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.706338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de5aa67a5be843d4a82687f850a55ca1', 'request_id': 'req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.715498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1033/2066] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:38 2022] DELETE /identity/v3/users/17fd0a0d0a9f4b61a57091684b86b80c => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:38.718884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:38.720284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:38.726301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] The token cache is being invalidate because project de5aa67a5be843d4a82687f850a55ca1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:38.726720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.727257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.729042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1034/2067] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:38 2022] DELETE /identity/v3/projects/de5aa67a5be843d4a82687f850a55ca1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:38.733621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:38.735369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:38.797978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef53429e-fba7-4e08-9417-7d37aa7383f5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:38.804028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:38.805625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:38.805899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:38.806292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] PATH_INFO: `/v3/projects/f7a8ad7892a64e55a74abd18a3d79450` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:38.807844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7a8ad7892a64e55a74abd18a3d79450)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:38.809835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:38.819182 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fafc3e5-93d4-4610-8f04-b8d4bb37ad4c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:38.825357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:38.826595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:38.826909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:38.827189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] PATH_INFO: `/v3/users/099101ec72eb4ef3b6287c514f0987ef` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:38.828953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] RBAC: Authorizing `identity:delete_user(user_id=099101ec72eb4ef3b6287c514f0987ef)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:38.830492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:38.880810 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7a8ad7892a64e55a74abd18a3d79450', 'request_id': 'req-37261170-0a66-43a2-b936-5750c49668c6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.888310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7a8ad7892a64e55a74abd18a3d79450', 'request_id': 'req-37261170-0a66-43a2-b936-5750c49668c6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.890214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '099101ec72eb4ef3b6287c514f0987ef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.893997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:38.898181 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7a8ad7892a64e55a74abd18a3d79450', 'request_id': 'req-37261170-0a66-43a2-b936-5750c49668c6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.900775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '099101ec72eb4ef3b6287c514f0987ef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.902679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '099101ec72eb4ef3b6287c514f0987ef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.911471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] The token cache is being invalidate because project f7a8ad7892a64e55a74abd18a3d79450 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:38.911884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.912680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.914246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1034/2068] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:38 2022] DELETE /identity/v3/projects/f7a8ad7892a64e55a74abd18a3d79450 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:38.919721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:38.921685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '099101ec72eb4ef3b6287c514f0987ef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.921811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:38.930934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '099101ec72eb4ef3b6287c514f0987ef'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:38.943681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1035/2069] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:38 2022] DELETE /identity/v3/users/099101ec72eb4ef3b6287c514f0987ef => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:39.019911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:39.021728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:39.043067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37261170-0a66-43a2-b936-5750c49668c6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:39.086675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:39.110853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af5250f-60fb-4db0-a154-f5fb42863509 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:39.118218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:39.119453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.120280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.120841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] PATH_INFO: `/v3/projects/a12c5d41c3ee49ddbee93cd8d7c3cc15` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.122553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a12c5d41c3ee49ddbee93cd8d7c3cc15)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:39.124892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:39.140756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:39.141904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.142298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.142563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] PATH_INFO: `/v3/users/70f4104c4d6642ae8ade42c669b4d643` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.143922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70f4104c4d6642ae8ade42c669b4d643)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:39.146015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:39.189173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a12c5d41c3ee49ddbee93cd8d7c3cc15', 'request_id': 'req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.200022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a12c5d41c3ee49ddbee93cd8d7c3cc15', 'request_id': 'req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.205511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70f4104c4d6642ae8ade42c669b4d643'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.208378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a12c5d41c3ee49ddbee93cd8d7c3cc15', 'request_id': 'req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.214499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70f4104c4d6642ae8ade42c669b4d643'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.215228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70f4104c4d6642ae8ade42c669b4d643'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.222813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70f4104c4d6642ae8ade42c669b4d643'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.223115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] The token cache is being invalidate because project a12c5d41c3ee49ddbee93cd8d7c3cc15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:39.223471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.224662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.230573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1036/2070] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:39 2022] DELETE /identity/v3/projects/a12c5d41c3ee49ddbee93cd8d7c3cc15 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:39.232330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:39.234072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:39.237207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70f4104c4d6642ae8ade42c669b4d643'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.244329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1035/2071] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:38 2022] DELETE /identity/v3/users/70f4104c4d6642ae8ade42c669b4d643 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:39.311297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e82d7-9d73-40e6-bad3-183d8f461bd8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:39.315301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:39.374526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:39.375423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.375713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.376018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] PATH_INFO: `/v3/users/e294bffa880c422ea82bc33fd75f46b7` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.377276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e294bffa880c422ea82bc33fd75f46b7)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:39.379459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:39.428455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e294bffa880c422ea82bc33fd75f46b7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.436287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e294bffa880c422ea82bc33fd75f46b7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.437255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e294bffa880c422ea82bc33fd75f46b7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.451594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e294bffa880c422ea82bc33fd75f46b7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.459820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cc1be5fb-299d-46f1-9386-d4e62e3a8a49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e294bffa880c422ea82bc33fd75f46b7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.466809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1037/2072] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:39 2022] DELETE /identity/v3/users/e294bffa880c422ea82bc33fd75f46b7 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:39.493119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:39.495292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:39.515614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dbc00-5ce1-448b-b4df-00eaea105347 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:39.600263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:39.602170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.602170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.602170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] PATH_INFO: `/v3/projects/6f20a3088a0f42c9af359dc131ef2b7f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.603245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f20a3088a0f42c9af359dc131ef2b7f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:39.606230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:39.679429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f20a3088a0f42c9af359dc131ef2b7f', 'request_id': 'req-cd951010-7a58-49c1-bc6a-024139d09b99'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.680575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:39.687052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f20a3088a0f42c9af359dc131ef2b7f', 'request_id': 'req-cd951010-7a58-49c1-bc6a-024139d09b99'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.701780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f20a3088a0f42c9af359dc131ef2b7f', 'request_id': 'req-cd951010-7a58-49c1-bc6a-024139d09b99'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.731247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] The token cache is being invalidate because project 6f20a3088a0f42c9af359dc131ef2b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:39.731576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.732337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.733792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1036/2073] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:39 2022] DELETE /identity/v3/projects/6f20a3088a0f42c9af359dc131ef2b7f => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:39.738918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f101e-645c-4b98-b77c-d8a65c7bdd30 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.739287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f101e-645c-4b98-b77c-d8a65c7bdd30 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.739635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957f101e-645c-4b98-b77c-d8a65c7bdd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.751964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:39.753953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:39.757732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-957f101e-645c-4b98-b77c-d8a65c7bdd30 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:39.759285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-957f101e-645c-4b98-b77c-d8a65c7bdd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:39.856826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1038/2074] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:39.864673 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77614d09-e7ab-4d15-ad66-88a9ecc4ed24 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.864963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77614d09-e7ab-4d15-ad66-88a9ecc4ed24 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.865245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77614d09-e7ab-4d15-ad66-88a9ecc4ed24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.866082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd951010-7a58-49c1-bc6a-024139d09b99 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:39.877004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:39.878072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.878348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.878619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] PATH_INFO: `/v3/projects/0f66a00b0bd14545abf17edc37ee7bf8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.879856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f66a00b0bd14545abf17edc37ee7bf8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:39.881807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:39.893260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-77614d09-e7ab-4d15-ad66-88a9ecc4ed24 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:39.894912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-77614d09-e7ab-4d15-ad66-88a9ecc4ed24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:39.905173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1039/2075] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:39.909138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77614d09-e7ab-4d15-ad66-88a9ecc4ed24 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:39.918539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77614d09-e7ab-4d15-ad66-88a9ecc4ed24 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:39.924306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:39.925184 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:39.925346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:39.925681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:39.926539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:39.926899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:39.928632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:39.943886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1040/2076] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:26:39.949876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:39.951631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f66a00b0bd14545abf17edc37ee7bf8', 'request_id': 'req-c1b79e66-e429-4425-b2e0-edd21e94d179'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.962569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7db6f-2f37-4c18-aacf-dce56170eb01 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:39.965570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f66a00b0bd14545abf17edc37ee7bf8', 'request_id': 'req-c1b79e66-e429-4425-b2e0-edd21e94d179'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:39.967265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:39.980179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f66a00b0bd14545abf17edc37ee7bf8', 'request_id': 'req-c1b79e66-e429-4425-b2e0-edd21e94d179'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.000492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] The token cache is being invalidate because project 0f66a00b0bd14545abf17edc37ee7bf8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:40.000647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.001423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.003207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1037/2077] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:39 2022] DELETE /identity/v3/projects/0f66a00b0bd14545abf17edc37ee7bf8 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:40.011043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.012605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.037172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a68228-2ecc-409c-9bd7-2a370c72486a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.037845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68228-2ecc-409c-9bd7-2a370c72486a admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.038136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68228-2ecc-409c-9bd7-2a370c72486a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.038437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a68228-2ecc-409c-9bd7-2a370c72486a admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.039206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a68228-2ecc-409c-9bd7-2a370c72486a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.041489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a68228-2ecc-409c-9bd7-2a370c72486a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.077494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1041/2078] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:39 2022] POST /identity/v3/projects => generated 385 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:40.077494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.077494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.077494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.084300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b79e66-e429-4425-b2e0-edd21e94d179 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.094665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:40.095909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.096359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:40.149727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.150659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.150947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.151252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] PATH_INFO: `/v3/projects/bde99b3a46cc4e059ee1965f24e1f30d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.152535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bde99b3a46cc4e059ee1965f24e1f30d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.154492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.164335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1042/2079] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:40.165155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.166481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.179987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e565065c-49c7-4442-a8cb-4a01471f9df0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.187389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa161c4-c1be-40ae-b224-5d8f5865f1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.188166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa161c4-c1be-40ae-b224-5d8f5865f1aa admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.188454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa161c4-c1be-40ae-b224-5d8f5865f1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.189236 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa161c4-c1be-40ae-b224-5d8f5865f1aa admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.190186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa161c4-c1be-40ae-b224-5d8f5865f1aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.192472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa161c4-c1be-40ae-b224-5d8f5865f1aa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.223040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bde99b3a46cc4e059ee1965f24e1f30d', 'request_id': 'req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.226446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1043/2080] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/users => generated 293 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:40.226446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012026f-dbf4-446f-b7b4-972a34af12de None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.226446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012026f-dbf4-446f-b7b4-972a34af12de None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.226714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b012026f-dbf4-446f-b7b4-972a34af12de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.230620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bde99b3a46cc4e059ee1965f24e1f30d', 'request_id': 'req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.240033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bde99b3a46cc4e059ee1965f24e1f30d', 'request_id': 'req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.249746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b012026f-dbf4-446f-b7b4-972a34af12de None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:40.251053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b012026f-dbf4-446f-b7b4-972a34af12de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.253002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] The token cache is being invalidate because project bde99b3a46cc4e059ee1965f24e1f30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:40.253993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.254694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:40.256517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1038/2081] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:40 2022] DELETE /identity/v3/projects/bde99b3a46cc4e059ee1965f24e1f30d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:40.257303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.258931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.331220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1044/2082] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:40.333157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c34c8c-d3cb-4bff-97be-84a8918645fd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.333627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c34c8c-d3cb-4bff-97be-84a8918645fd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.333917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c34c8c-d3cb-4bff-97be-84a8918645fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.363012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-56c34c8c-d3cb-4bff-97be-84a8918645fd None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:40.363012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-56c34c8c-d3cb-4bff-97be-84a8918645fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.382330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1045/2083] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:40.382330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c34c8c-d3cb-4bff-97be-84a8918645fd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.396573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c34c8c-d3cb-4bff-97be-84a8918645fd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.405254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6adb5-3c9f-4249-b3b0-65d3f6573234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.405949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6adb5-3c9f-4249-b3b0-65d3f6573234 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.406231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6adb5-3c9f-4249-b3b0-65d3f6573234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.406511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6adb5-3c9f-4249-b3b0-65d3f6573234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.407279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6adb5-3c9f-4249-b3b0-65d3f6573234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:40.407603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6adb5-3c9f-4249-b3b0-65d3f6573234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.409762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6adb5-3c9f-4249-b3b0-65d3f6573234 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.413286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:40.418626 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc7219-9a3e-4913-9916-4a9c8a83fc39 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.428943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.431596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.431769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1046/2084] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:40.432142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.433378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.434432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368a297-871e-47bb-a7c8-eac8bf0feea4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.434903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368a297-871e-47bb-a7c8-eac8bf0feea4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.435028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.435195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a368a297-871e-47bb-a7c8-eac8bf0feea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.437797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.457554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a368a297-871e-47bb-a7c8-eac8bf0feea4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:40.461481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1039/2085] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:40.461481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.463057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a368a297-871e-47bb-a7c8-eac8bf0feea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.473073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9217bc94-977e-4d0f-b710-a448a1dbffc8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.476241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1047/2086] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:40.477511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368a297-871e-47bb-a7c8-eac8bf0feea4 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.482856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.483543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.483851 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.484133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.484836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.486961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.492574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a368a297-871e-47bb-a7c8-eac8bf0feea4 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.498936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.499359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.499644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.499976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.500713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.511874 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.528824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1048/2087] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:40.532767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.545358 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea76b8d-958d-430d-903e-d906d412fad1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.552491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.553257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.553562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.553847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.554615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:40.554933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.556789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.557184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1040/2088] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:40.558945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.576942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1049/2089] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:40.581507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.605386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189d228-7019-4dba-8cf0-d3f7945c6553 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.606336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:40.655912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc139c8a-a90d-4fad-83ca-529e480d0581 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.662257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.663640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.664121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.664461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] PATH_INFO: `/v3/projects/d439bc0bd723450e8af866cc50b46d85/users/3e6ad5a28c8341f19244e8d935deb6d1/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.678318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.679515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.679959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.680249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.681120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.682083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d439bc0bd723450e8af866cc50b46d85, user_id=3e6ad5a28c8341f19244e8d935deb6d1, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.683859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.685464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.707397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1041/2090] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:40 2022] PUT /identity/v3/projects/d439bc0bd723450e8af866cc50b46d85/users/3e6ad5a28c8341f19244e8d935deb6d1/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:40.708594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.711037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1050/2091] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/users => generated 291 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:40.712731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.723092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f16209f-be15-4ab5-b0cb-bde8da3d98f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.800775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3181a2a0-b963-4f72-9213-97a7b65e1ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.801458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181a2a0-b963-4f72-9213-97a7b65e1ac4 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.801774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181a2a0-b963-4f72-9213-97a7b65e1ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.802051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3181a2a0-b963-4f72-9213-97a7b65e1ac4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.802781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181a2a0-b963-4f72-9213-97a7b65e1ac4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.804397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3005f-edbf-45b3-89fa-f41bc787237c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.804519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3181a2a0-b963-4f72-9213-97a7b65e1ac4 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.810465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.811323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.812095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.812095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.812764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.814826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.815861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1051/2092] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:40.816974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91cb3-909b-4522-abd0-299e96f8762b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.817265 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91cb3-909b-4522-abd0-299e96f8762b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.817539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e91cb3-909b-4522-abd0-299e96f8762b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.846550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-46e91cb3-909b-4522-abd0-299e96f8762b None None] MFA Rules not processed for user `3e6ad5a28c8341f19244e8d935deb6d1`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:40.847645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1042/2093] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/projects => generated 383 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:40.847892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-46e91cb3-909b-4522-abd0-299e96f8762b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:40.851706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.903574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:40.928163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca95ccff-bdd9-47c0-a689-b7317e43b639 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.931953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1052/2094] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:40.931953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e91cb3-909b-4522-abd0-299e96f8762b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.934033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.934885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.935172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.935452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.936223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.937834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.944000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e91cb3-909b-4522-abd0-299e96f8762b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.949392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.949936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1043/2095] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:40 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:40.950748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.952338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.952783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.953283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.954072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.960431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbecfc2-bbe6-430a-ae6f-5fd90edb445f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:40.960745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.965428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:40.966088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:40.966360 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:40.966685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:40.967361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:40.969541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:40.983779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1053/2096] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/users => generated 292 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:40.987501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:40.997790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b665d0b-9f9d-4f24-8766-4e73558fb9ea admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.004539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1044/2097] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:40 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:41.004539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.005745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.006499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.006797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.007076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] PATH_INFO: `/v3/projects/e8c25cc48b4145e3953b91d6bff6ea6c/users/26b721f42d7f4044b75cffc72fbed994/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.009701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:41.023126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8c25cc48b4145e3953b91d6bff6ea6c, user_id=26b721f42d7f4044b75cffc72fbed994, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.025818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.041746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1054/2098] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:40 2022] PUT /identity/v3/projects/e8c25cc48b4145e3953b91d6bff6ea6c/users/26b721f42d7f4044b75cffc72fbed994/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:41.043263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.104057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b5e49c-263d-4de0-ba76-269c2d67f5e2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.116811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a8599-2ba4-446b-9081-3e03d1368215 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.121475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc78fdaa-a6bd-47ea-8a7c-4e5df47c1e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.122146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78fdaa-a6bd-47ea-8a7c-4e5df47c1e69 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.122459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78fdaa-a6bd-47ea-8a7c-4e5df47c1e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.122737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc78fdaa-a6bd-47ea-8a7c-4e5df47c1e69 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.123627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78fdaa-a6bd-47ea-8a7c-4e5df47c1e69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.125379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc78fdaa-a6bd-47ea-8a7c-4e5df47c1e69 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.146978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1055/2099] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:41 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:41.148256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480038a1-508a-40ff-b187-b645b6371ef7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.148544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480038a1-508a-40ff-b187-b645b6371ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.148821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480038a1-508a-40ff-b187-b645b6371ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.167911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.168662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.169014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.169354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.170147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.171861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.180008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-480038a1-508a-40ff-b187-b645b6371ef7 None None] MFA Rules not processed for user `26b721f42d7f4044b75cffc72fbed994`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:41.181420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-480038a1-508a-40ff-b187-b645b6371ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:41.182581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1045/2100] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:41.183470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.195356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad07eb6a-ffc3-4e2f-aed9-7447b6fb2f98 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.200742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.201610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.201983 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.202325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.203232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.205105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.237558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1046/2101] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:41.238697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.257087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35f398-6283-4c12-a17c-1b07757a6462 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.272527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.273834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.274366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.275117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.277876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.281984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.299452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1047/2102] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:41.300446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.314323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6a7f87-c923-47d3-865a-8a4f7e502a1a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.322234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.323789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.324134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.324420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] PATH_INFO: `/v3/projects/1e148c73c113492ab3e24c4953169a87/users/c6af603c97464e7880a62fad3e0636bd/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.364660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1056/2103] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:41.365557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480038a1-508a-40ff-b187-b645b6371ef7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.397718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e148c73c113492ab3e24c4953169a87, user_id=c6af603c97464e7880a62fad3e0636bd, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.410534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.414033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480038a1-508a-40ff-b187-b645b6371ef7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.422040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.423490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.424476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.425008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] PATH_INFO: `/v3/projects/d3e4c9a9eb054dd8834df686b4430c1b/users/b9faa9cee4b248d8b219fb1a359014bf/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.429677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1048/2104] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:41 2022] PUT /identity/v3/projects/1e148c73c113492ab3e24c4953169a87/users/c6af603c97464e7880a62fad3e0636bd/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:41.431170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.437172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:41.451905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e4c9a9eb054dd8834df686b4430c1b, user_id=b9faa9cee4b248d8b219fb1a359014bf, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.454887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.458284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:41.483596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1057/2105] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:41 2022] PUT /identity/v3/projects/d3e4c9a9eb054dd8834df686b4430c1b/users/b9faa9cee4b248d8b219fb1a359014bf/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:41.485553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.528274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a564b-ff87-4981-9e09-3c3f6c5cc235 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.541977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:41.582665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c345a449-a2bf-4570-9413-657018101cda admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.585995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.586722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.587075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.587423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.587532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0649e6e3-fd44-493b-9a7b-bc916777cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.588366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649e6e3-fd44-493b-9a7b-bc916777cb0f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.589218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649e6e3-fd44-493b-9a7b-bc916777cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.589502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0649e6e3-fd44-493b-9a7b-bc916777cb0f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.590398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649e6e3-fd44-493b-9a7b-bc916777cb0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.592039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0649e6e3-fd44-493b-9a7b-bc916777cb0f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.606749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1058/2106] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:41 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:41.606749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2a5e4-c2c8-4bd6-acb6-0b3dfb72bbd9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.606749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2a5e4-c2c8-4bd6-acb6-0b3dfb72bbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.606749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b2a5e4-c2c8-4bd6-acb6-0b3dfb72bbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.634471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-41b2a5e4-c2c8-4bd6-acb6-0b3dfb72bbd9 None None] MFA Rules not processed for user `b9faa9cee4b248d8b219fb1a359014bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:41.636525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41b2a5e4-c2c8-4bd6-acb6-0b3dfb72bbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:41.657078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.658797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.671391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1049/2107] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:41 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:41.672414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.689265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5b5e5-826d-49e3-b969-ce9600188f49 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.696091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.696999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.697284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.697566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] PATH_INFO: `/v3/projects/1e148c73c113492ab3e24c4953169a87/users/c6af603c97464e7880a62fad3e0636bd/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.699795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e148c73c113492ab3e24c4953169a87, user_id=c6af603c97464e7880a62fad3e0636bd, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.703958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.716068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1050/2108] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:41 2022] PUT /identity/v3/projects/1e148c73c113492ab3e24c4953169a87/users/c6af603c97464e7880a62fad3e0636bd/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:41.716812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.731324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1059/2109] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:41.732735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.733031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.733308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.763043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] MFA Rules not processed for user `c6af603c97464e7880a62fad3e0636bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:41.764291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:41.793066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2b0f29-cb16-4e2e-9fbc-a0af4d235593 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.798448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.800134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.800134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.800314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] PATH_INFO: `/v3/users/26b721f42d7f4044b75cffc72fbed994` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.801763 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b721f42d7f4044b75cffc72fbed994)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:41.803894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:41.839235 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1060/2110] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:41.840118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.841404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:41.848148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:41.852602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b721f42d7f4044b75cffc72fbed994'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:41.860717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b721f42d7f4044b75cffc72fbed994'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:41.861406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b721f42d7f4044b75cffc72fbed994'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:41.866778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:41.869619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b721f42d7f4044b75cffc72fbed994'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:41.883711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b721f42d7f4044b75cffc72fbed994'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:41.890627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1051/2111] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:41 2022] DELETE /identity/v3/users/26b721f42d7f4044b75cffc72fbed994 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:41.891572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:41.892967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:41.905577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a86d19-b94d-4d18-b418-937678d1eaec None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.969246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c387996-c070-46af-b966-af4a66c54f4a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:41.975477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.976769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.977406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.977694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.981813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:41.985164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fdade2-0e10-4269-b762-01552401ab79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:41.985934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdade2-0e10-4269-b762-01552401ab79 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:41.986229 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdade2-0e10-4269-b762-01552401ab79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:41.986593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fdade2-0e10-4269-b762-01552401ab79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:41.988687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-30fdade2-0e10-4269-b762-01552401ab79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:42.077361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:42.079888 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:42.100971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdade2-0e10-4269-b762-01552401ab79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:42.102440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fdade2-0e10-4269-b762-01552401ab79 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:42.130684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1061/2112] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:42.133398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1052/2113] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:42.134064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:42.134365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:42.139409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:42.198061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:42.198061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:42.284444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1062/2114] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:42.294775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:42.296204 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:42.409288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7750b372-8874-4a53-a7be-e6df05a909c9 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:42.417397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bf4bdb-fc39-46fa-83e2-466cf56640a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:42.418105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf4bdb-fc39-46fa-83e2-466cf56640a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:42.418726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf4bdb-fc39-46fa-83e2-466cf56640a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:42.419006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bf4bdb-fc39-46fa-83e2-466cf56640a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:42.436360 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf4bdb-fc39-46fa-83e2-466cf56640a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:42.437910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bf4bdb-fc39-46fa-83e2-466cf56640a3 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:42.460716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1053/2115] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:42.854773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:42.857328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:42.891231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e9496b-d504-4abf-aced-ac03af0633b7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:42.914157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:42.916295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:42.916594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:42.917122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] PATH_INFO: `/v3/projects/e8c25cc48b4145e3953b91d6bff6ea6c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:42.919210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8c25cc48b4145e3953b91d6bff6ea6c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:42.923676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:42.959435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:43.065162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8c25cc48b4145e3953b91d6bff6ea6c', 'request_id': 'req-05254543-6fcb-415f-a72d-506effe11f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.085700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8c25cc48b4145e3953b91d6bff6ea6c', 'request_id': 'req-05254543-6fcb-415f-a72d-506effe11f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.094594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8c25cc48b4145e3953b91d6bff6ea6c', 'request_id': 'req-05254543-6fcb-415f-a72d-506effe11f94'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.115867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] The token cache is being invalidate because project e8c25cc48b4145e3953b91d6bff6ea6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:43.116382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.117119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05254543-6fcb-415f-a72d-506effe11f94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.122560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1063/2116] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:42 2022] DELETE /identity/v3/projects/e8c25cc48b4145e3953b91d6bff6ea6c => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:43.124552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8335f1-50ea-4f61-92de-7f4e050ea70b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.125002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8335f1-50ea-4f61-92de-7f4e050ea70b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.125342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8335f1-50ea-4f61-92de-7f4e050ea70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.145955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0e8335f1-50ea-4f61-92de-7f4e050ea70b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:43.147259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0e8335f1-50ea-4f61-92de-7f4e050ea70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:43.177792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:43.249971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1054/2117] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:43.260027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f5a75-9c61-45d5-83f4-4c864233bf56 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.261691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f5a75-9c61-45d5-83f4-4c864233bf56 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.262554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f5a75-9c61-45d5-83f4-4c864233bf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.306923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-802f5a75-9c61-45d5-83f4-4c864233bf56 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:43.310612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-802f5a75-9c61-45d5-83f4-4c864233bf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:43.332064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1064/2118] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:43.338122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8335f1-50ea-4f61-92de-7f4e050ea70b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.355542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8335f1-50ea-4f61-92de-7f4e050ea70b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:43.366558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:43.369623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.370607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.372283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.374416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:43.376100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:43.377857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:43.411051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1055/2119] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:43.411627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f5a75-9c61-45d5-83f4-4c864233bf56 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.428520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f5a75-9c61-45d5-83f4-4c864233bf56 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:43.448625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:43.449482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.449772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.450051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.450770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:43.452730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:43.518011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1065/2120] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:43 2022] POST /identity/v3/projects => generated 377 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:43.527030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.539605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7981e651-b64d-4e08-b8eb-28e22934ff74 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:43.618329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:43.619033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.619673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.619673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.620335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:43.622259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:43.642883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1056/2121] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:43 2022] POST /identity/v3/users => generated 289 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:43.646801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.655145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.656888 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:43.658152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0c0798-6b26-4610-954b-5192b724ed22 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:43.667856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:43.669077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.669299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63856da-0e4c-4c47-9b1c-fdb5deb7b4c9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:43.669610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.670106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.671494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:43.675390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:43.675652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:43.677294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.678108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.678496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] PATH_INFO: `/v3/users/3e6ad5a28c8341f19244e8d935deb6d1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.682141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e6ad5a28c8341f19244e8d935deb6d1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:43.685986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:43.694018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1066/2122] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:43.695565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.709608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b667568-11db-413d-bf4d-90678c20bc32 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:43.716647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:43.717480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.718335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.718723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.720170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:43.722844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:43.735834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1067/2123] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:43 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:43.740009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.754226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c91db-90ba-4ee1-9bbb-7e768775afc2 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:43.759028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca16ac30-e111-4d6f-a60c-1cda708e8a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:43.759968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16ac30-e111-4d6f-a60c-1cda708e8a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.760260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16ac30-e111-4d6f-a60c-1cda708e8a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.761080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca16ac30-e111-4d6f-a60c-1cda708e8a72 admin admin] PATH_INFO: `/v3/projects/9f1d5f362a4d410d846ddf2e919da8cb/users/f66cb5f2ec024a1f97e1c2fa47cc2dc1/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.764266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e6ad5a28c8341f19244e8d935deb6d1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.773378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e6ad5a28c8341f19244e8d935deb6d1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.774106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e6ad5a28c8341f19244e8d935deb6d1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.779875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16ac30-e111-4d6f-a60c-1cda708e8a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f1d5f362a4d410d846ddf2e919da8cb, user_id=f66cb5f2ec024a1f97e1c2fa47cc2dc1, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:43.782241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e6ad5a28c8341f19244e8d935deb6d1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.782528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca16ac30-e111-4d6f-a60c-1cda708e8a72 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:43.797521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1068/2124] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:43 2022] PUT /identity/v3/projects/9f1d5f362a4d410d846ddf2e919da8cb/users/f66cb5f2ec024a1f97e1c2fa47cc2dc1/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:43.802004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ca1d973e-70d7-4120-a25f-1ebcdc248ffd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e6ad5a28c8341f19244e8d935deb6d1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:43.807725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.807963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.808244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.812559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1057/2125] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:43 2022] DELETE /identity/v3/users/3e6ad5a28c8341f19244e8d935deb6d1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:43.815001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:43.815243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:43.815833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:43.836569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:43.837779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:43.840709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] MFA Rules not processed for user `f66cb5f2ec024a1f97e1c2fa47cc2dc1`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:43.841947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:43.850502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:43.923543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1058/2126] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:43.923850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.925332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:43.929913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1069/2127] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:43 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:43.943966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:43.945966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:43.964513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:44.002580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c87b37-2ce2-4464-848c-5c64858c2d05 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:44.009392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:44.010124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:44.010998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:44.011239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:44.020840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:44.022283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:44.031112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7c71f7-9e04-4a80-936d-3d776c345a04 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:44.032255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1059/2128] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:44.035193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:44.035915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:44.036202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:44.036480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:44.046395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:44.047864 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:44.057764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1070/2129] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:44.333480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:44.335197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:44.350924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220672f-c2e1-4d0d-b510-b6da0624b0d7 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:44.357556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:44.358435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:44.358739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:44.358969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] PATH_INFO: `/v3/projects/d439bc0bd723450e8af866cc50b46d85` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:44.360249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d439bc0bd723450e8af866cc50b46d85)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:44.362168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:44.413985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd439bc0bd723450e8af866cc50b46d85', 'request_id': 'req-915a5907-379b-425c-943d-8c5d1c2440d2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.421033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd439bc0bd723450e8af866cc50b46d85', 'request_id': 'req-915a5907-379b-425c-943d-8c5d1c2440d2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.434036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd439bc0bd723450e8af866cc50b46d85', 'request_id': 'req-915a5907-379b-425c-943d-8c5d1c2440d2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.444657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] The token cache is being invalidate because project d439bc0bd723450e8af866cc50b46d85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:44.445221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.445836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.447460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1060/2130] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:44 2022] DELETE /identity/v3/projects/d439bc0bd723450e8af866cc50b46d85 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:44.451558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:44.465509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf6d3c-b6fd-4c4c-b6ba-4721bd3fecc7 service glance] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:44.528274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:44.529394 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:44.529678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:44.529953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] PATH_INFO: `/v3/users/c6af603c97464e7880a62fad3e0636bd` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:44.531511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6af603c97464e7880a62fad3e0636bd)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:44.534476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:44.633585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6af603c97464e7880a62fad3e0636bd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.642053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6af603c97464e7880a62fad3e0636bd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.648168 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6af603c97464e7880a62fad3e0636bd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.664348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6af603c97464e7880a62fad3e0636bd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.673772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-fdef1032-8201-4f45-a9ca-718a94fe5c2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6af603c97464e7880a62fad3e0636bd'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:44.680931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1071/2131] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:44 2022] DELETE /identity/v3/users/c6af603c97464e7880a62fad3e0636bd => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:45.090325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.091877 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:45.105023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a5907-379b-425c-943d-8c5d1c2440d2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.147216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:45.166607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.167506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.167805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.168080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] PATH_INFO: `/v3/projects/1e148c73c113492ab3e24c4953169a87` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.169278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e148c73c113492ab3e24c4953169a87)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.171220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.222498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e148c73c113492ab3e24c4953169a87', 'request_id': 'req-4b214315-ccdb-4920-9049-08a9f7757af5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:45.228631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e148c73c113492ab3e24c4953169a87', 'request_id': 'req-4b214315-ccdb-4920-9049-08a9f7757af5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:45.236987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e148c73c113492ab3e24c4953169a87', 'request_id': 'req-4b214315-ccdb-4920-9049-08a9f7757af5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:45.248929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] The token cache is being invalidate because project 1e148c73c113492ab3e24c4953169a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:45.249370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:45.250127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4b214315-ccdb-4920-9049-08a9f7757af5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:45.252107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1061/2132] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:45 2022] DELETE /identity/v3/projects/1e148c73c113492ab3e24c4953169a87 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:45.258204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3898085-dd98-478a-8a72-f53ebb015db9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.258659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3898085-dd98-478a-8a72-f53ebb015db9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.258766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3898085-dd98-478a-8a72-f53ebb015db9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.277609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a3898085-dd98-478a-8a72-f53ebb015db9 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:45.278811 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3898085-dd98-478a-8a72-f53ebb015db9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:45.312996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:45.346646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1072/2133] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:45.354526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672297a0-d41b-480d-956a-c2923d8f157c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.355053 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672297a0-d41b-480d-956a-c2923d8f157c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.355168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672297a0-d41b-480d-956a-c2923d8f157c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.374571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-672297a0-d41b-480d-956a-c2923d8f157c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:45.375826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-672297a0-d41b-480d-956a-c2923d8f157c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:45.384528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1062/2134] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:45.389501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3898085-dd98-478a-8a72-f53ebb015db9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.399665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3898085-dd98-478a-8a72-f53ebb015db9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.404582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.405314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.405604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.405895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.406619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:45.406934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.408589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.421514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1073/2135] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:45.425742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672297a0-d41b-480d-956a-c2923d8f157c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.438060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672297a0-d41b-480d-956a-c2923d8f157c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.443135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.443933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.444289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.444636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.445672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.447679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.477612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1063/2136] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:45 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:45.481714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.491530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a627c0b-e193-4aad-865d-a6c360d7e037 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.555411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.556409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.556696 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.556974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.558222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.560201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.584483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1074/2137] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:45 2022] POST /identity/v3/users => generated 285 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:45.588325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.599088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47dda54-657d-4b5e-abbd-5967a040a829 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.604071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.604814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.605201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.605495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.606301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.608050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.625785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1064/2138] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:26:45.627955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.639939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fec4853-c9db-49ff-8a0f-7a662c308cf5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.646557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8d0ae5-a7fa-44b7-b054-233a934c280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.647222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0ae5-a7fa-44b7-b054-233a934c280c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.647582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0ae5-a7fa-44b7-b054-233a934c280c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.647953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8d0ae5-a7fa-44b7-b054-233a934c280c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.648757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d0ae5-a7fa-44b7-b054-233a934c280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.650431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8d0ae5-a7fa-44b7-b054-233a934c280c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.664482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1075/2139] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:45.673012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.683687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2249a-9a0c-4158-8dc9-ff5a009449f3 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.688304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.689297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.689589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.689870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] PATH_INFO: `/v3/projects/cf8af4dee7e84c5f9f61deaea32ff163/users/1d26db3bcf154dd98e5e62dc47a41dd9/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.707577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf8af4dee7e84c5f9f61deaea32ff163, user_id=1d26db3bcf154dd98e5e62dc47a41dd9, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.710307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.724582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1065/2140] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:45 2022] PUT /identity/v3/projects/cf8af4dee7e84c5f9f61deaea32ff163/users/1d26db3bcf154dd98e5e62dc47a41dd9/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:45.735352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.735758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.736113 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.763735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] MFA Rules not processed for user `1d26db3bcf154dd98e5e62dc47a41dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:45.765150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:45.832506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1076/2141] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:45.843808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:45.845595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:45.903612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5511e4d4-af0e-43d5-aaeb-59daf3124a23 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:45.907730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4e3f4e-af84-497a-99e4-3cd7a3a3fa6b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:45.908411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e3f4e-af84-497a-99e4-3cd7a3a3fa6b service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:45.908688 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e3f4e-af84-497a-99e4-3cd7a3a3fa6b service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:45.908966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4e3f4e-af84-497a-99e4-3cd7a3a3fa6b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:45.920406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e3f4e-af84-497a-99e4-3cd7a3a3fa6b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:45.921841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4e3f4e-af84-497a-99e4-3cd7a3a3fa6b service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:45.931407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1066/2142] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:45 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:51.060207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:51.061662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:51.099518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:51.130617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6513f348-6d75-476b-8f9e-e404b3d0420d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:51.135088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:51.136534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:51.136822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:51.137103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] PATH_INFO: `/v3/users/ea6bde782a1447daa0b6d7b6960162ce` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:51.138306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea6bde782a1447daa0b6d7b6960162ce)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:51.140262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:51.188222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea6bde782a1447daa0b6d7b6960162ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.196640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea6bde782a1447daa0b6d7b6960162ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.197393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea6bde782a1447daa0b6d7b6960162ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.212470 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea6bde782a1447daa0b6d7b6960162ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.220856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea6bde782a1447daa0b6d7b6960162ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.227947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1077/2143] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:51 2022] DELETE /identity/v3/users/ea6bde782a1447daa0b6d7b6960162ce => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:51.232134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:51.232627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:51.232903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:51.250956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:51.252226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:51.261440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:51.329353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1067/2144] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:51.336419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:51.338202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:51.439907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e782c5a8-b1a1-46d8-8e04-4315e60058af admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:51.444687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:51.445621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:51.445910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:51.446244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:51.459044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:51.460672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:51.475986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1078/2145] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:51.845811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:51.847795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:51.862009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef14a771-36c1-4821-aa17-9eab7a397261 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:51.867209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:51.868241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:51.868689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:51.869504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] PATH_INFO: `/v3/projects/1dc6117c79124a1aab8844d5c1dc7e0d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:51.871537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc6117c79124a1aab8844d5c1dc7e0d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:51.874270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:51.923597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc6117c79124a1aab8844d5c1dc7e0d', 'request_id': 'req-dd74ea91-1895-42c4-bc81-e71c284a159b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.929862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc6117c79124a1aab8844d5c1dc7e0d', 'request_id': 'req-dd74ea91-1895-42c4-bc81-e71c284a159b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.937472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc6117c79124a1aab8844d5c1dc7e0d', 'request_id': 'req-dd74ea91-1895-42c4-bc81-e71c284a159b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.949459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] The token cache is being invalidate because project 1dc6117c79124a1aab8844d5c1dc7e0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:51.949711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.950452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:51.951947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1068/2146] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:51 2022] DELETE /identity/v3/projects/1dc6117c79124a1aab8844d5c1dc7e0d => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:51.955937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:51.966258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f105fd-406e-4c40-9fb0-fcf52a9cf578 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:52.022612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:52.040098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:52.041244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:52.042272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:52.042554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] PATH_INFO: `/v3/users/97962bffb29c4bebb8b6e4c2f7383018` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:52.045406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97962bffb29c4bebb8b6e4c2f7383018)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:52.047415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:52.095052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97962bffb29c4bebb8b6e4c2f7383018'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.102211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97962bffb29c4bebb8b6e4c2f7383018'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.102831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97962bffb29c4bebb8b6e4c2f7383018'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.115312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97962bffb29c4bebb8b6e4c2f7383018'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.122863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97962bffb29c4bebb8b6e4c2f7383018'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.129418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1079/2147] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:51 2022] DELETE /identity/v3/users/97962bffb29c4bebb8b6e4c2f7383018 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:52.394735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:52.396209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:52.411169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74ea91-1895-42c4-bc81-e71c284a159b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:52.469083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:52.469949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:52.470235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:52.470513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] PATH_INFO: `/v3/projects/80cb94d89e4241e091bb5bbd4aaea6c3` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:52.471716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] RBAC: Authorizing `identity:delete_project(project_id=80cb94d89e4241e091bb5bbd4aaea6c3)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:52.473682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:52.517791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:52.525542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80cb94d89e4241e091bb5bbd4aaea6c3', 'request_id': 'req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.532654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80cb94d89e4241e091bb5bbd4aaea6c3', 'request_id': 'req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.539676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80cb94d89e4241e091bb5bbd4aaea6c3', 'request_id': 'req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.549664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] The token cache is being invalidate because project 80cb94d89e4241e091bb5bbd4aaea6c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:52.549952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.550570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.552136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1069/2148] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:52 2022] DELETE /identity/v3/projects/80cb94d89e4241e091bb5bbd4aaea6c3 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:52.556264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:52.557729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:52.570713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1fc73c6-a9ac-4cd3-80ca-617309a12769 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:52.626297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:52.627192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:52.627476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:52.627764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] PATH_INFO: `/v3/users/8c2326faea9d4a569089cb5dbbf558ce` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:52.628959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c2326faea9d4a569089cb5dbbf558ce)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:52.630869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:52.677766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c2326faea9d4a569089cb5dbbf558ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.684761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c2326faea9d4a569089cb5dbbf558ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.685617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c2326faea9d4a569089cb5dbbf558ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.700415 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c2326faea9d4a569089cb5dbbf558ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.702625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:52.708720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-cb82c829-a2cc-47bd-a89c-69b41c9ef2ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c2326faea9d4a569089cb5dbbf558ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:52.717290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1080/2149] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:52 2022] DELETE /identity/v3/users/8c2326faea9d4a569089cb5dbbf558ce => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:52.984289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:52.985722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:52.999468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe7f4d-fbc2-469f-bc47-42a7eb66a63d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:53.060107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:53.061063 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.061555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.062084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] PATH_INFO: `/v3/projects/64773073271c4ef9b6fd893d4505175b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.063627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=64773073271c4ef9b6fd893d4505175b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:53.065572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:53.116214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64773073271c4ef9b6fd893d4505175b', 'request_id': 'req-3cdea846-6145-462e-9ee0-f95499ba46ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:53.122870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64773073271c4ef9b6fd893d4505175b', 'request_id': 'req-3cdea846-6145-462e-9ee0-f95499ba46ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:53.130319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64773073271c4ef9b6fd893d4505175b', 'request_id': 'req-3cdea846-6145-462e-9ee0-f95499ba46ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:53.141685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] The token cache is being invalidate because project 64773073271c4ef9b6fd893d4505175b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:53.142004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:53.142643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3cdea846-6145-462e-9ee0-f95499ba46ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:53.144210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1070/2150] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:52 2022] DELETE /identity/v3/projects/64773073271c4ef9b6fd893d4505175b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:53.149019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ab7f3-8147-4c77-8a3b-bee71db6b14b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.149483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ab7f3-8147-4c77-8a3b-bee71db6b14b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.149576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ab7f3-8147-4c77-8a3b-bee71db6b14b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.169002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a70ab7f3-8147-4c77-8a3b-bee71db6b14b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:53.170405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a70ab7f3-8147-4c77-8a3b-bee71db6b14b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:53.235416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1081/2151] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:53.243205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85a186-c25d-43a3-8dcb-1f339b2dd9ed None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.243605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85a186-c25d-43a3-8dcb-1f339b2dd9ed None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.243975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d85a186-c25d-43a3-8dcb-1f339b2dd9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.263000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3d85a186-c25d-43a3-8dcb-1f339b2dd9ed None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:53.264243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3d85a186-c25d-43a3-8dcb-1f339b2dd9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:53.272167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1071/2152] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:53.276353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ab7f3-8147-4c77-8a3b-bee71db6b14b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:53.286002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ab7f3-8147-4c77-8a3b-bee71db6b14b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:53.290513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:53.291165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.292005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.292286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.293058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:53.293318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:53.294902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:53.307719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1082/2153] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:53.311878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85a186-c25d-43a3-8dcb-1f339b2dd9ed None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:53.321601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d85a186-c25d-43a3-8dcb-1f339b2dd9ed None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:53.326493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:53.327142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.327424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.327699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.328441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:53.330339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:53.355155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1072/2154] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:53 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:53.359472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:53.369262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a123b4-2ece-4cf5-b2ae-db460e8743ce admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:53.425351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:53.426011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.426299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.426578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.427275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:53.429220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:53.449850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1083/2155] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:53 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:53.453917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:53.464609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b906c9b-2c49-4050-9029-cb92e9be3c38 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:53.469325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:53.469981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.470262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.470534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.471255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:53.472878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:53.483635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1073/2156] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:53.487416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:53.497891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b77f2e-554a-4680-9a9f-5289bea4eeea admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:53.502524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e1c03-0331-437f-957b-27a2fc2445cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:53.503186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e1c03-0331-437f-957b-27a2fc2445cb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.503478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e1c03-0331-437f-957b-27a2fc2445cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.503779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e1c03-0331-437f-957b-27a2fc2445cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.504516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e1c03-0331-437f-957b-27a2fc2445cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:53.506120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e1c03-0331-437f-957b-27a2fc2445cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:53.519814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1084/2157] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:53.521699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:53.531573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df94e3a-a777-45c4-ab6c-4a53a8a325bd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:53.536230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:53.537099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.537381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.537657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] PATH_INFO: `/v3/projects/17f33598fdb14edaa577b108c1ea87fa/users/83047d45a7f74e468e4635ecb391cf7d/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.552404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=17f33598fdb14edaa577b108c1ea87fa, user_id=83047d45a7f74e468e4635ecb391cf7d, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:53.554971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:53.557785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:53.569481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1074/2158] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:53 2022] PUT /identity/v3/projects/17f33598fdb14edaa577b108c1ea87fa/users/83047d45a7f74e468e4635ecb391cf7d/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:53.580887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539a76a-b54a-4f2a-b810-accffa3341df None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:53.581184 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539a76a-b54a-4f2a-b810-accffa3341df None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:53.581502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b539a76a-b54a-4f2a-b810-accffa3341df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:53.610796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b539a76a-b54a-4f2a-b810-accffa3341df None None] MFA Rules not processed for user `83047d45a7f74e468e4635ecb391cf7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:53.612028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b539a76a-b54a-4f2a-b810-accffa3341df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:53.657164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:53.677701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1085/2159] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:53 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:57.127981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:57.129666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:57.198974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f06f264-1b48-4dca-b037-652134230de4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:57.203521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:57.204446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:57.204729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:57.205003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] PATH_INFO: `/v3/users/b9faa9cee4b248d8b219fb1a359014bf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:57.206185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9faa9cee4b248d8b219fb1a359014bf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:57.208192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:57.256016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9faa9cee4b248d8b219fb1a359014bf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.263903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9faa9cee4b248d8b219fb1a359014bf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.264522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9faa9cee4b248d8b219fb1a359014bf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.271590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9faa9cee4b248d8b219fb1a359014bf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.285178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9faa9cee4b248d8b219fb1a359014bf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.297201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1075/2160] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:57 2022] DELETE /identity/v3/users/b9faa9cee4b248d8b219fb1a359014bf => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:57.301642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:57.302159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:57.302271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:57.321215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:57.322444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:57.381677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1086/2161] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:57.391271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:57.392676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:57.454253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55d084-9c3b-4648-be93-b0de6dc54caa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:57.458345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53aeb74-80aa-4db9-b826-fe46daeb7b14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:57.458999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aeb74-80aa-4db9-b826-fe46daeb7b14 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:57.459277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aeb74-80aa-4db9-b826-fe46daeb7b14 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:57.459552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53aeb74-80aa-4db9-b826-fe46daeb7b14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:57.469622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53aeb74-80aa-4db9-b826-fe46daeb7b14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:57.471051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53aeb74-80aa-4db9-b826-fe46daeb7b14 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:57.480986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1076/2162] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:26:57.741570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:57.743112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:57.759159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee81f85-42c0-400b-875b-1dbc8122ce7c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:57.766907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:57.767955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:57.768411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:57.768848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] PATH_INFO: `/v3/projects/d3e4c9a9eb054dd8834df686b4430c1b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:57.770907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e4c9a9eb054dd8834df686b4430c1b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:57.775083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:57.833370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e4c9a9eb054dd8834df686b4430c1b', 'request_id': 'req-d2d7fe4c-a252-488a-b101-3e3623a726bc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.841419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e4c9a9eb054dd8834df686b4430c1b', 'request_id': 'req-d2d7fe4c-a252-488a-b101-3e3623a726bc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.848047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e4c9a9eb054dd8834df686b4430c1b', 'request_id': 'req-d2d7fe4c-a252-488a-b101-3e3623a726bc'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.861959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] The token cache is being invalidate because project d3e4c9a9eb054dd8834df686b4430c1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:57.862253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.863535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d2d7fe4c-a252-488a-b101-3e3623a726bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:57.865417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1087/2163] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:57 2022] DELETE /identity/v3/projects/d3e4c9a9eb054dd8834df686b4430c1b => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:57.870504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea658-3868-4788-b5c2-84d7729b3ec3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:57.870825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea658-3868-4788-b5c2-84d7729b3ec3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:57.871306 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140ea658-3868-4788-b5c2-84d7729b3ec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:57.892977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-140ea658-3868-4788-b5c2-84d7729b3ec3 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:57.894198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-140ea658-3868-4788-b5c2-84d7729b3ec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:57.910423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:57.968307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1077/2164] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:57.977071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5674a2-c16a-447d-b03f-23a3e0a67d9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:57.977531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5674a2-c16a-447d-b03f-23a3e0a67d9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:57.977909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5674a2-c16a-447d-b03f-23a3e0a67d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:57.999913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6b5674a2-c16a-447d-b03f-23a3e0a67d9d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:58.001283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b5674a2-c16a-447d-b03f-23a3e0a67d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:58.012288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1088/2165] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:58.014346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea658-3868-4788-b5c2-84d7729b3ec3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.025498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140ea658-3868-4788-b5c2-84d7729b3ec3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.030531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.031198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.031479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.031770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.032506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:26:58.033189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.034541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.047099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1078/2166] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:26:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:26:58.051494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5674a2-c16a-447d-b03f-23a3e0a67d9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.061394 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5674a2-c16a-447d-b03f-23a3e0a67d9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.066153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.066840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.067126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.067407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.068158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.070331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.095311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1089/2167] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:58 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:58.099800 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.111850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d450348-03c5-40fb-b16d-4b7a7e0cdb2b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.170396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.171118 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.171447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.171799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.172581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.174671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.193658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1079/2168] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:58 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:58.198148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.208601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96f3eab-40fb-43ba-8765-1dd305fc0b91 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.214152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.214822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.215108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.215386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.216205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.217808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.229334 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1090/2169] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:58.233237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.243045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea08fe7-ca5b-4df6-8d00-94e875b1145b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.247696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3d4baf-b020-42d1-b1f3-1d974c878a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.248387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4baf-b020-42d1-b1f3-1d974c878a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.248672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4baf-b020-42d1-b1f3-1d974c878a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.248947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3d4baf-b020-42d1-b1f3-1d974c878a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.249662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d4baf-b020-42d1-b1f3-1d974c878a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.251267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3d4baf-b020-42d1-b1f3-1d974c878a8e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.262582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1080/2170] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:58.267797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.278730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f8136-3388-4685-b7b9-8693046b18e3 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.283704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.284658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.284951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.285257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] PATH_INFO: `/v3/projects/bb01d4c4d6c14223b746c3e2b6a0b4ce/users/8e65287b73044b3c9e724fe6993983d5/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.301549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb01d4c4d6c14223b746c3e2b6a0b4ce, user_id=8e65287b73044b3c9e724fe6993983d5, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.304577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.320860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1091/2171] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:58 2022] PUT /identity/v3/projects/bb01d4c4d6c14223b746c3e2b6a0b4ce/users/8e65287b73044b3c9e724fe6993983d5/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:58.331150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2d644-90bc-4bcd-92a3-fa4bd45d1f53 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.331443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2d644-90bc-4bcd-92a3-fa4bd45d1f53 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.331728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2d644-90bc-4bcd-92a3-fa4bd45d1f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.363685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d2c2d644-90bc-4bcd-92a3-fa4bd45d1f53 None None] MFA Rules not processed for user `8e65287b73044b3c9e724fe6993983d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:58.365119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2c2d644-90bc-4bcd-92a3-fa4bd45d1f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:58.433707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1081/2172] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:58.436096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.449832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:58.510276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b696d6-2a9e-4ab4-837a-a0032660f3c9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.515050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.515717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.516036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.516318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.517021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.518952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.544545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1092/2173] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:26:58 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:26:58.548883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2d644-90bc-4bcd-92a3-fa4bd45d1f53 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.560477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c2d644-90bc-4bcd-92a3-fa4bd45d1f53 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.571468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:58.622093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.622766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.623052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.623331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.624059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.625989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.647261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1082/2174] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:26:58 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:26:58.651373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.661620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904a605-2815-4014-9748-afd2c9aad245 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.667002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.667665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.667989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.668270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.668999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.670594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.680983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1093/2175] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:58.685709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.695787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd23a29-3ed0-4b4c-b954-c931d0a4f4be admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.700618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.701552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.701836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.702112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] PATH_INFO: `/v3/projects/52750a8870564c439b3b1356d119c832/users/692ad7899af940ed96b7eb6fd01cc777/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.718249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52750a8870564c439b3b1356d119c832, user_id=692ad7899af940ed96b7eb6fd01cc777, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.720974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.733815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1083/2176] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:58 2022] PUT /identity/v3/projects/52750a8870564c439b3b1356d119c832/users/692ad7899af940ed96b7eb6fd01cc777/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:58.737943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.749264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8703f221-3cca-4fce-89ef-e749dc5b0a08 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.810775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21055d85-8a52-474e-8a8d-91e7501fa373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.811594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055d85-8a52-474e-8a8d-91e7501fa373 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.811829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055d85-8a52-474e-8a8d-91e7501fa373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.812111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21055d85-8a52-474e-8a8d-91e7501fa373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.812841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21055d85-8a52-474e-8a8d-91e7501fa373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.814704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21055d85-8a52-474e-8a8d-91e7501fa373 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.828572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1094/2177] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:26:58 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:26:58.829691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.842067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596cd2c-4034-47bc-b7b2-19403d90dff6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:58.847320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:58.848282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.848569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.848858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] PATH_INFO: `/v3/projects/52750a8870564c439b3b1356d119c832/users/692ad7899af940ed96b7eb6fd01cc777/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.850960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52750a8870564c439b3b1356d119c832, user_id=692ad7899af940ed96b7eb6fd01cc777, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:58.854543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:58.866869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1084/2178] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:26:58 2022] PUT /identity/v3/projects/52750a8870564c439b3b1356d119c832/users/692ad7899af940ed96b7eb6fd01cc777/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:58.877999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:58.878288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:58.878566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:58.908379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] MFA Rules not processed for user `692ad7899af940ed96b7eb6fd01cc777`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:58.909601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:58.929776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:58.931568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:58.990337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1095/2179] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:58 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:26:59.002567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:59.003959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:59.009128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:59.009725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6e8380-8dbe-4c53-aa4a-06423d3e4064 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:59.016590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:59.017480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:59.017745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:59.018029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] PATH_INFO: `/v3/users/614857eb4afe4b3da58d083731a1b683` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:59.019231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=614857eb4afe4b3da58d083731a1b683)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:59.021185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:59.084428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '614857eb4afe4b3da58d083731a1b683'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.089720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80abc2c9-a3ff-4b69-87aa-d51e15650f43 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:59.094042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '614857eb4afe4b3da58d083731a1b683'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.094708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '614857eb4afe4b3da58d083731a1b683'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.095036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:59.095722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:59.096135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:59.096420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:59.098399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:59.102928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '614857eb4afe4b3da58d083731a1b683'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.105551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:INSERT=6,keystone:DELETE=8 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:59.124961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5778aa88-42ad-4ac0-bc13-52e721c3dd75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '614857eb4afe4b3da58d083731a1b683'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.131945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1085/2180] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:58 2022] DELETE /identity/v3/users/614857eb4afe4b3da58d083731a1b683 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:26:59.136865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:59.137228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:59.137771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:59.157144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:26:59.158243 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:59.168619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:59.170063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:59.182196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1096/2181] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:59.224552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1086/2182] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:26:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:26:59.235127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:59.236432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:59.300258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e67ae8-46e7-469d-9530-e4ab8369e7cc service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:59.355593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:59.356544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:59.356717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:59.357303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:59.374562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:59.376490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:59.389947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1097/2183] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:26:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:26:59.710308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:59.711870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:26:59.725006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9bdfa1-6f30-4adc-967a-1ac35ac3df4a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:59.730352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:59.731339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:59.731723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:59.732189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] PATH_INFO: `/v3/projects/59bfb77cec7c4192909af3515a730817` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:59.733491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59bfb77cec7c4192909af3515a730817)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:59.735687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:59.791397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59bfb77cec7c4192909af3515a730817', 'request_id': 'req-94955f93-2aa4-42e2-9f32-dba4edff8ad9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.797599 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59bfb77cec7c4192909af3515a730817', 'request_id': 'req-94955f93-2aa4-42e2-9f32-dba4edff8ad9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.805164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59bfb77cec7c4192909af3515a730817', 'request_id': 'req-94955f93-2aa4-42e2-9f32-dba4edff8ad9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.818023 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] The token cache is being invalidate because project 59bfb77cec7c4192909af3515a730817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:26:59.818526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.819176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.821060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1087/2184] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:26:59 2022] DELETE /identity/v3/projects/59bfb77cec7c4192909af3515a730817 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:26:59.825872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:26:59.835672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33484de-bd8f-47f6-a9a3-cbcdd8c9f67a service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:26:59.847325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:26:59.901995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:26:59.902883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:26:59.903165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:26:59.903441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] PATH_INFO: `/v3/users/f583a83f0d974512855096dffc5914c0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:26:59.904690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f583a83f0d974512855096dffc5914c0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:26:59.906655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:26:59.957085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f583a83f0d974512855096dffc5914c0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.963591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f583a83f0d974512855096dffc5914c0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.964294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f583a83f0d974512855096dffc5914c0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.976538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f583a83f0d974512855096dffc5914c0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.983718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4b07aba0-bb3c-4ce0-b012-4269cdd49ed7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f583a83f0d974512855096dffc5914c0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:26:59.989660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1098/2185] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:26:59 2022] DELETE /identity/v3/users/f583a83f0d974512855096dffc5914c0 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:00.287138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:00.288717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:00.313923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94955f93-2aa4-42e2-9f32-dba4edff8ad9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:00.376577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:00.377602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.377882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.378210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] PATH_INFO: `/v3/projects/79d7b3ca71a94af3833f39cf98e233b8` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.379408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d7b3ca71a94af3833f39cf98e233b8)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:00.381436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:00.389840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:00.433632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d7b3ca71a94af3833f39cf98e233b8', 'request_id': 'req-20000f31-6630-4046-80bf-1479db7fece7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:00.440182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d7b3ca71a94af3833f39cf98e233b8', 'request_id': 'req-20000f31-6630-4046-80bf-1479db7fece7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:00.452180 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d7b3ca71a94af3833f39cf98e233b8', 'request_id': 'req-20000f31-6630-4046-80bf-1479db7fece7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:00.462819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] The token cache is being invalidate because project 79d7b3ca71a94af3833f39cf98e233b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:00.463611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:00.464286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-20000f31-6630-4046-80bf-1479db7fece7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:00.465810 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1088/2186] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:00 2022] DELETE /identity/v3/projects/79d7b3ca71a94af3833f39cf98e233b8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:00.470914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4361c60-fb91-4507-b0f8-01023c8d7c63 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.471418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4361c60-fb91-4507-b0f8-01023c8d7c63 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.471623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4361c60-fb91-4507-b0f8-01023c8d7c63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.490420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-f4361c60-fb91-4507-b0f8-01023c8d7c63 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:00.491655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f4361c60-fb91-4507-b0f8-01023c8d7c63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:00.562459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1099/2187] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:00.572168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299db3b0-a01f-4e0f-9c2f-c933244c7c37 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.572605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299db3b0-a01f-4e0f-9c2f-c933244c7c37 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.572888 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299db3b0-a01f-4e0f-9c2f-c933244c7c37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.592078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-299db3b0-a01f-4e0f-9c2f-c933244c7c37 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:00.593338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-299db3b0-a01f-4e0f-9c2f-c933244c7c37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:00.601358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1089/2188] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:00.606239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4361c60-fb91-4507-b0f8-01023c8d7c63 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:00.616017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4361c60-fb91-4507-b0f8-01023c8d7c63 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:00.620614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:00.621270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.621658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.621821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.628675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:00.628675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:00.628675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:00.638992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1100/2189] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:00.643317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299db3b0-a01f-4e0f-9c2f-c933244c7c37 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:00.653678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299db3b0-a01f-4e0f-9c2f-c933244c7c37 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:00.658500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:00.659156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.659441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.659715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.660442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:00.662355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:00.692422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1090/2190] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:00 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:00.696606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:00.707786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bfd7eb-fd78-4da3-ae5e-92c9c3bc16b7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:00.727173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:00.774113 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:00.774714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.775002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.775284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.776631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:00.778545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:00.797440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1101/2191] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:00 2022] POST /identity/v3/users => generated 284 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:00.802155 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:00.813433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424817f9-fb3d-4e24-b16d-3b82214c51b1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:00.818097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:00.818767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.819052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.819323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.820337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:00.821928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:00.833186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1091/2192] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:00.837326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:00.848033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999e67c-66f6-40bb-9c65-245f972a4595 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:00.852695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e188307-00fa-45cd-92bd-079b5ca902eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:00.853352 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e188307-00fa-45cd-92bd-079b5ca902eb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.853642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e188307-00fa-45cd-92bd-079b5ca902eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.853922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e188307-00fa-45cd-92bd-079b5ca902eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.854653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e188307-00fa-45cd-92bd-079b5ca902eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:00.856296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e188307-00fa-45cd-92bd-079b5ca902eb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:00.866378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1102/2193] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:00.870904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:00.880989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617e2f1-2d08-4d26-bccc-fb0f69d8db8e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:00.886786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:00.887676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.887996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.888278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] PATH_INFO: `/v3/projects/f2fd7a74ff484be78c7025efb216fa64/users/83d21b9051184aa79e8a6ac189d993f0/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.903759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2fd7a74ff484be78c7025efb216fa64, user_id=83d21b9051184aa79e8a6ac189d993f0, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:00.906439 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:00.918882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1092/2194] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:00 2022] PUT /identity/v3/projects/f2fd7a74ff484be78c7025efb216fa64/users/83d21b9051184aa79e8a6ac189d993f0/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:00.929261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18b2a-0717-4fa1-979e-7aec619220b2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:00.929561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18b2a-0717-4fa1-979e-7aec619220b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:00.929845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d18b2a-0717-4fa1-979e-7aec619220b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:00.957678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-c7d18b2a-0717-4fa1-979e-7aec619220b2 None None] MFA Rules not processed for user `83d21b9051184aa79e8a6ac189d993f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:00.958899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c7d18b2a-0717-4fa1-979e-7aec619220b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:01.027871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1103/2195] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:00 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:01.032856 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.117618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e98cf5-7deb-446c-bf0c-94db548b4df8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.123193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.124005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.124316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.124664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.125434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.127487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.159079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1093/2196] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:01 2022] POST /identity/v3/projects => generated 367 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:27:01.163514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d18b2a-0717-4fa1-979e-7aec619220b2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.174744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d18b2a-0717-4fa1-979e-7aec619220b2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.230136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.230815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.231106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.231385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.232128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.234063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.253455 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1104/2197] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:01 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:01.257445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.268785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d9fa4-bf51-417c-b2ae-70b67bfe0407 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.273904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.274587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.274875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.275158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.276079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.277782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.288936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1094/2198] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:01.293104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.303190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e2fd85-e365-4d81-8cb4-37141ecd9d45 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.308657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.309584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.309909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.310197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] PATH_INFO: `/v3/projects/d0e4dceee85e4bf1bfae3f12deda9d65/users/de814bfe1e3541d39fc502833c703c22/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.326232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e4dceee85e4bf1bfae3f12deda9d65, user_id=de814bfe1e3541d39fc502833c703c22, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.328931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.332044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:01.345053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1105/2199] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:01 2022] PUT /identity/v3/projects/d0e4dceee85e4bf1bfae3f12deda9d65/users/de814bfe1e3541d39fc502833c703c22/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:01.349082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.360121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8d1107-9c7b-4ecf-970a-d5d41c7fb308 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.364112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:01.421339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcafea4c-6d27-4897-8c01-940d56eecebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.421999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafea4c-6d27-4897-8c01-940d56eecebd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.422278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafea4c-6d27-4897-8c01-940d56eecebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.422550 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcafea4c-6d27-4897-8c01-940d56eecebd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.423263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafea4c-6d27-4897-8c01-940d56eecebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.425049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcafea4c-6d27-4897-8c01-940d56eecebd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.435304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1095/2200] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:01.439717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.451286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85400f0-813c-4ca5-8748-86db037ff37e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.455942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.457796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.457796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.457796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] PATH_INFO: `/v3/projects/d0e4dceee85e4bf1bfae3f12deda9d65/users/de814bfe1e3541d39fc502833c703c22/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.461897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0e4dceee85e4bf1bfae3f12deda9d65, user_id=de814bfe1e3541d39fc502833c703c22, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.464818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.481664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1106/2201] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:01 2022] PUT /identity/v3/projects/d0e4dceee85e4bf1bfae3f12deda9d65/users/de814bfe1e3541d39fc502833c703c22/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:01.492789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e99e3-d942-4efb-9590-8757c77701f0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.493336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e99e3-d942-4efb-9590-8757c77701f0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.493852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92e99e3-d942-4efb-9590-8757c77701f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.535200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d92e99e3-d942-4efb-9590-8757c77701f0 None None] MFA Rules not processed for user `de814bfe1e3541d39fc502833c703c22`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:01.536625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d92e99e3-d942-4efb-9590-8757c77701f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:01.617889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1096/2202] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:01.630826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.632618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:01.690439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93284fca-16fb-43e1-9e20-9f1cbad13802 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.694144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.694790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.695330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.695615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.704619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.706044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.715413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1107/2203] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:01.758528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e99e3-d942-4efb-9590-8757c77701f0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.766463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:01.767908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:01.768825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92e99e3-d942-4efb-9590-8757c77701f0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.772714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb90de0-b634-4859-9032-a15f2218095e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.773581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90de0-b634-4859-9032-a15f2218095e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.773870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90de0-b634-4859-9032-a15f2218095e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.774157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb90de0-b634-4859-9032-a15f2218095e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.838779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb90de0-b634-4859-9032-a15f2218095e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.840285 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb90de0-b634-4859-9032-a15f2218095e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.841398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1939d4c-cdd8-453d-b821-5f08aa6c39b7 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:01.846906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:01.848459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.848969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.849491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] PATH_INFO: `/v3/users/8e65287b73044b3c9e724fe6993983d5` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.850417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1097/2204] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:01 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:01.852081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e65287b73044b3c9e724fe6993983d5)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:01.856547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:01.912217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e65287b73044b3c9e724fe6993983d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:01.920023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e65287b73044b3c9e724fe6993983d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:01.920603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e65287b73044b3c9e724fe6993983d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:01.933122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e65287b73044b3c9e724fe6993983d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:01.939957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e65287b73044b3c9e724fe6993983d5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:01.946252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1108/2205] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:01 2022] DELETE /identity/v3/users/8e65287b73044b3c9e724fe6993983d5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:01.951032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:01.951323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:01.951605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:01.971580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:01.973015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:01.997269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:02.043624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1098/2206] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:02.053301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:02.055175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:02.063458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:02.161670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6c0e3f-2b9a-4b6f-a8cb-c24f0d3032c8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:02.169623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:02.170380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:02.170729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:02.171076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:02.198412 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:02.200189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:02.211781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1109/2207] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:02.989458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:02.990887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:03.004308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae8ca2-d496-4f8c-8f50-fbcc767015b0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:03.010845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:03.011948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:03.012299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:03.012640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] PATH_INFO: `/v3/projects/bb01d4c4d6c14223b746c3e2b6a0b4ce` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:03.014166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb01d4c4d6c14223b746c3e2b6a0b4ce)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:03.015995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:03.099172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb01d4c4d6c14223b746c3e2b6a0b4ce', 'request_id': 'req-6e2df788-387d-448e-a79c-82e70599abd1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.106606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb01d4c4d6c14223b746c3e2b6a0b4ce', 'request_id': 'req-6e2df788-387d-448e-a79c-82e70599abd1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.116915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb01d4c4d6c14223b746c3e2b6a0b4ce', 'request_id': 'req-6e2df788-387d-448e-a79c-82e70599abd1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.138269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] The token cache is being invalidate because project bb01d4c4d6c14223b746c3e2b6a0b4ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:03.138840 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.139600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.142553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1099/2208] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:02 2022] DELETE /identity/v3/projects/bb01d4c4d6c14223b746c3e2b6a0b4ce => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:03.146152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:03.158311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8b10ea-abdc-4dd3-829b-408d25969316 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:03.245627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:03.246506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:03.246788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:03.247063 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] PATH_INFO: `/v3/users/692ad7899af940ed96b7eb6fd01cc777` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:03.248286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=692ad7899af940ed96b7eb6fd01cc777)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:03.250309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:03.300332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '692ad7899af940ed96b7eb6fd01cc777'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.309840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '692ad7899af940ed96b7eb6fd01cc777'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.310659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '692ad7899af940ed96b7eb6fd01cc777'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.328865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '692ad7899af940ed96b7eb6fd01cc777'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.336602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8f8aa754-818a-4228-91d1-83452cb10084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '692ad7899af940ed96b7eb6fd01cc777'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.343225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1110/2209] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:03 2022] DELETE /identity/v3/users/692ad7899af940ed96b7eb6fd01cc777 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:03.728125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:03.740654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:03.760112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2df788-387d-448e-a79c-82e70599abd1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:03.844850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:03.845744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:03.846041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:03.846323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] PATH_INFO: `/v3/projects/52750a8870564c439b3b1356d119c832` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:03.848055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52750a8870564c439b3b1356d119c832)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:03.850467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:03.898745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52750a8870564c439b3b1356d119c832', 'request_id': 'req-4ceac054-21e5-4df2-866c-49d4d3d69ed4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.901164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:03.905732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52750a8870564c439b3b1356d119c832', 'request_id': 'req-4ceac054-21e5-4df2-866c-49d4d3d69ed4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.913187 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52750a8870564c439b3b1356d119c832', 'request_id': 'req-4ceac054-21e5-4df2-866c-49d4d3d69ed4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.924753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] The token cache is being invalidate because project 52750a8870564c439b3b1356d119c832 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:03.925040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.925798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-4ceac054-21e5-4df2-866c-49d4d3d69ed4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:03.928690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1100/2210] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:03 2022] DELETE /identity/v3/projects/52750a8870564c439b3b1356d119c832 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:03.935183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b58c0-e31c-4e69-9a74-b3794087800c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:03.935477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b58c0-e31c-4e69-9a74-b3794087800c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:03.935790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59b58c0-e31c-4e69-9a74-b3794087800c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:03.956445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b59b58c0-e31c-4e69-9a74-b3794087800c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:03.957667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b59b58c0-e31c-4e69-9a74-b3794087800c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:03.980586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:04.036336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1111/2211] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:04.045228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766300b-746e-4762-a1c5-05274ddc513c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.046655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766300b-746e-4762-a1c5-05274ddc513c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.046989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766300b-746e-4762-a1c5-05274ddc513c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.068161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a766300b-746e-4762-a1c5-05274ddc513c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:04.069381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a766300b-746e-4762-a1c5-05274ddc513c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:04.079043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1101/2212] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:04.083011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b58c0-e31c-4e69-9a74-b3794087800c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:04.093555 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59b58c0-e31c-4e69-9a74-b3794087800c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:04.101661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:04.102527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.102878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.103221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.104782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:04.105187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:04.106887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:04.122694 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1112/2213] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:04.126968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766300b-746e-4762-a1c5-05274ddc513c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:04.136639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766300b-746e-4762-a1c5-05274ddc513c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:04.141891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:04.142569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.142833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.143109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.143999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:04.145918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:04.182374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1102/2214] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:04 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:04.186909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:04.197984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de732df-c1f2-4198-ac4e-065c6cf98e9b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:04.255643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:04.256562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.256857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.257250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.258056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:04.260069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:04.283466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1113/2215] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:04 2022] POST /identity/v3/users => generated 293 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:04.284635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:04.295623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076c96a1-6569-4d65-8f72-6e9340d4f0ed admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:04.300160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:04.300807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.301084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.301354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.302061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:04.303652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:04.315785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1103/2216] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:04.319385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:04.329797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3541da1-8a3b-4bf1-befc-c80091bc8fdf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:04.334798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adcfcbb4-cdc7-4424-a79a-4c5ac4f675cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:04.335492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfcbb4-cdc7-4424-a79a-4c5ac4f675cb admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.335821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfcbb4-cdc7-4424-a79a-4c5ac4f675cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.336104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcfcbb4-cdc7-4424-a79a-4c5ac4f675cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.336843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfcbb4-cdc7-4424-a79a-4c5ac4f675cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:04.339000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adcfcbb4-cdc7-4424-a79a-4c5ac4f675cb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:04.350398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1114/2217] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:04.355045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:04.365393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01caf40a-0e09-4477-9219-3e194140604c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:04.370971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:04.371879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.372162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.372437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] PATH_INFO: `/v3/projects/5a2481546251413fb65aaed9e3f131e2/users/9ae3a6ce6ab4425ebf1180721c10eb2d/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.386503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a2481546251413fb65aaed9e3f131e2, user_id=9ae3a6ce6ab4425ebf1180721c10eb2d, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:04.389643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:04.402649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1104/2218] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:04 2022] PUT /identity/v3/projects/5a2481546251413fb65aaed9e3f131e2/users/9ae3a6ce6ab4425ebf1180721c10eb2d/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:04.408102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.408417 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.408681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.429262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:04.430568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:04.493067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1115/2219] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:04.509676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:04.513721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:04.574682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489928ef-a943-4931-9032-f864408b52bd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:04.578395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948843c8-0c81-4186-9fdc-b9a9ba225e28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:04.579062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948843c8-0c81-4186-9fdc-b9a9ba225e28 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:04.579344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948843c8-0c81-4186-9fdc-b9a9ba225e28 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:04.579616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948843c8-0c81-4186-9fdc-b9a9ba225e28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:04.589938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948843c8-0c81-4186-9fdc-b9a9ba225e28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:04.591371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948843c8-0c81-4186-9fdc-b9a9ba225e28 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:04.603285 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1105/2220] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:05.472125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:05.473534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:05.486176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04559e0-4cce-4e91-8af1-5ebc968b1553 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:05.491090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:05.492070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:05.492361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:05.492647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] PATH_INFO: `/v3/users/83d21b9051184aa79e8a6ac189d993f0` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:05.493866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=83d21b9051184aa79e8a6ac189d993f0)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:05.496516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:05.528925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:05.567424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83d21b9051184aa79e8a6ac189d993f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:05.576416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83d21b9051184aa79e8a6ac189d993f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:05.577925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83d21b9051184aa79e8a6ac189d993f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:05.593589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83d21b9051184aa79e8a6ac189d993f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:05.602892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83d21b9051184aa79e8a6ac189d993f0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:05.611636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1116/2221] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:05 2022] DELETE /identity/v3/users/83d21b9051184aa79e8a6ac189d993f0 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:05.617127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:05.618077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:05.618567 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:05.637198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:05.638514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:05.714776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1106/2222] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:05.726727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:05.729486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:05.802600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d65201d-12ac-4e36-a8d8-882fdfbf58bc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:05.806665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:05.807925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:05.807925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:05.808217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:05.819518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:05.821261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:05.833878 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1117/2223] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:06.533423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:06.535802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:06.554289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6bb591-6f5b-492e-935a-08a1b380acf7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:06.560436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:06.560914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:06.561507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:06.561878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] PATH_INFO: `/v3/projects/f2fd7a74ff484be78c7025efb216fa64` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:06.564375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2fd7a74ff484be78c7025efb216fa64)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:06.571341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:06.604171 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=94,keystone:INSERT=3,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:06.697736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2fd7a74ff484be78c7025efb216fa64', 'request_id': 'req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:06.712519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2fd7a74ff484be78c7025efb216fa64', 'request_id': 'req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:06.732744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2fd7a74ff484be78c7025efb216fa64', 'request_id': 'req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:06.780408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] The token cache is being invalidate because project f2fd7a74ff484be78c7025efb216fa64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:06.780748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:06.782482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:06.784789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1107/2224] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:06 2022] DELETE /identity/v3/projects/f2fd7a74ff484be78c7025efb216fa64 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:06.789025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:06.801135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e54ada-198f-4431-afd9-55f56683c4e8 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:06.897100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:06.898292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:06.898644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:06.898987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] PATH_INFO: `/v3/users/de814bfe1e3541d39fc502833c703c22` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:06.901111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] RBAC: Authorizing `identity:delete_user(user_id=de814bfe1e3541d39fc502833c703c22)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:06.903138 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:06.969707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de814bfe1e3541d39fc502833c703c22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:06.973272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:06.981896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de814bfe1e3541d39fc502833c703c22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:06.982712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de814bfe1e3541d39fc502833c703c22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:07.000025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de814bfe1e3541d39fc502833c703c22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:07.015801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-54bf6f7d-4dc2-4cd9-bba6-bdf3a919adff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de814bfe1e3541d39fc502833c703c22'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:07.026418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1118/2225] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:06 2022] DELETE /identity/v3/users/de814bfe1e3541d39fc502833c703c22 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:07.878211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:07.883478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:07.907011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c4a621-1018-446f-b3bb-c870dbcd0cd6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:08.055963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:08.057013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:08.057548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:08.058029 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] PATH_INFO: `/v3/projects/d0e4dceee85e4bf1bfae3f12deda9d65` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:08.063318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0e4dceee85e4bf1bfae3f12deda9d65)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:08.065562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:08.171713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0e4dceee85e4bf1bfae3f12deda9d65', 'request_id': 'req-46eb6f92-43ef-477e-8241-d92f008b5287'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:08.191611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0e4dceee85e4bf1bfae3f12deda9d65', 'request_id': 'req-46eb6f92-43ef-477e-8241-d92f008b5287'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:08.213289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0e4dceee85e4bf1bfae3f12deda9d65', 'request_id': 'req-46eb6f92-43ef-477e-8241-d92f008b5287'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:08.230305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] The token cache is being invalidate because project d0e4dceee85e4bf1bfae3f12deda9d65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:08.232956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:08.233899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46eb6f92-43ef-477e-8241-d92f008b5287 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:08.237491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1108/2226] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:07 2022] DELETE /identity/v3/projects/d0e4dceee85e4bf1bfae3f12deda9d65 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:08.242021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731566-ab57-418a-b650-662f12aa06ac None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:08.242822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731566-ab57-418a-b650-662f12aa06ac None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:08.243299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c731566-ab57-418a-b650-662f12aa06ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:08.267933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c731566-ab57-418a-b650-662f12aa06ac None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:08.269581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c731566-ab57-418a-b650-662f12aa06ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:08.353078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1119/2227] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:08.362356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89356b66-1a16-4abf-b68b-4cf59741e782 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:08.362879 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89356b66-1a16-4abf-b68b-4cf59741e782 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:08.363383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89356b66-1a16-4abf-b68b-4cf59741e782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:08.407893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-89356b66-1a16-4abf-b68b-4cf59741e782 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:08.413167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-89356b66-1a16-4abf-b68b-4cf59741e782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:08.438601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731566-ab57-418a-b650-662f12aa06ac None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:08.446143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1109/2228] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:08.453608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c731566-ab57-418a-b650-662f12aa06ac None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:08.473500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:08.474213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:08.474903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:08.475202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:08.476021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:08.476343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:08.478018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:08.503635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1120/2229] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:08.517193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89356b66-1a16-4abf-b68b-4cf59741e782 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:08.569859 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89356b66-1a16-4abf-b68b-4cf59741e782 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:08.592611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:08.593678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:08.594185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:08.594669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:08.595595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:08.598018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:08.636488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1110/2230] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:08 2022] POST /identity/v3/projects => generated 359 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:08.640996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:08.673856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242ce8e5-02f4-4aad-b6cf-6b4a1143fe4b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:08.816724 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:08.817700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:08.818003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:08.828452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:08.829439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:08.831772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:08.874179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1121/2231] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:08 2022] POST /identity/v3/users => generated 280 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:08.876649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:08.884279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:08.893051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959e2344-2942-4982-a59a-e30b948f3e0b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:08.909450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:08.912667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:08.912954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:08.913233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:08.913982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:08.917341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:08.947779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1111/2232] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:08.958428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:08.979522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce9f35e-4d5d-4c74-951a-23f854a21445 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:08.997307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c256d07-4e9c-40b5-9c10-2a15b7bc3470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:08.998034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c256d07-4e9c-40b5-9c10-2a15b7bc3470 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:09.000338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c256d07-4e9c-40b5-9c10-2a15b7bc3470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:09.000706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c256d07-4e9c-40b5-9c10-2a15b7bc3470 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:09.004795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c256d07-4e9c-40b5-9c10-2a15b7bc3470 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:09.025057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c256d07-4e9c-40b5-9c10-2a15b7bc3470 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:09.052671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:09.053699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1122/2233] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:08 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:09.065017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bcbbba-79fb-4270-866f-009e88668c93 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:09.078360 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:09.079503 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:09.080043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:09.080453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] PATH_INFO: `/v3/projects/be3af6e5ba244de3bb4f16570a4f42e3/users/ed9a9d0aef7f4fa4969b405d68993bce/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:09.123981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] RBAC: Authorizing `identity:create_grant(project_id=be3af6e5ba244de3bb4f16570a4f42e3, user_id=ed9a9d0aef7f4fa4969b405d68993bce, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:09.127028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:09.157299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1112/2234] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:09 2022] PUT /identity/v3/projects/be3af6e5ba244de3bb4f16570a4f42e3/users/ed9a9d0aef7f4fa4969b405d68993bce/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:09.167809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:09.168182 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:09.168478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:09.230015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] MFA Rules not processed for user `ed9a9d0aef7f4fa4969b405d68993bce`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:09.231303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:09.289759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:09.438953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1123/2235] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:09 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:09.479586 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:09.484811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:09.589017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:09.590741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:09.612747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dcc234-3bad-4189-afa0-efa75959eefc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:09.629639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:09.630662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:09.630662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:09.630957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:09.647146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:09.648732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:09.673562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1113/2236] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:09 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:09.689321 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:09.737189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a19d23-f467-4303-88ba-a81f37d6ce48 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:09.791553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:09.793675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:09.795282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:09.796613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:09.807575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:09.837629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a84ab1-19c8-43c1-b85f-11cf10ee9260 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:09.858603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:09.859506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:09.859827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:09.860110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] PATH_INFO: `/v3/users/ed9a9d0aef7f4fa4969b405d68993bce` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:09.867205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed9a9d0aef7f4fa4969b405d68993bce)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:09.869787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:09.959478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed9a9d0aef7f4fa4969b405d68993bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:09.971052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed9a9d0aef7f4fa4969b405d68993bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:09.972051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed9a9d0aef7f4fa4969b405d68993bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:09.983383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed9a9d0aef7f4fa4969b405d68993bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:09.998015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:10.015867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0592ebf8-c5f3-4595-b3ad-b8cdfd124749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed9a9d0aef7f4fa4969b405d68993bce'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:10.028088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1114/2237] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:09 2022] DELETE /identity/v3/users/ed9a9d0aef7f4fa4969b405d68993bce => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:10.030521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f46077-dee1-4feb-8e34-e414f5249ca6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:10.030961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f46077-dee1-4feb-8e34-e414f5249ca6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:10.031348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f46077-dee1-4feb-8e34-e414f5249ca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:10.035145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:10.036767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:10.053985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1124/2238] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:09 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:27:10.057281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-68f46077-dee1-4feb-8e34-e414f5249ca6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:10.059676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-68f46077-dee1-4feb-8e34-e414f5249ca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:10.198613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1115/2239] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:10.251123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:10.252609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:10.457158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292658f-ac0e-4971-a82a-b77fd3e48630 service glance] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:10.523494 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:10.553542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:10.554457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:10.554860 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:10.555336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:10.566703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:10.568697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:10.590588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1125/2240] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:10.769472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726cdeb-a498-4fe5-b5b6-27edd1bdb3b3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:10.769764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726cdeb-a498-4fe5-b5b6-27edd1bdb3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:10.770046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b726cdeb-a498-4fe5-b5b6-27edd1bdb3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:10.816147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b726cdeb-a498-4fe5-b5b6-27edd1bdb3b3 None None] MFA Rules not processed for user `9ae3a6ce6ab4425ebf1180721c10eb2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:10.817802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b726cdeb-a498-4fe5-b5b6-27edd1bdb3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:10.999610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1116/2241] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:11.001716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.003892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:11.023643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fd0f2-7c71-4080-bd3f-548fa62721c2 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.044574 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.045529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.045929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.046219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.046954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.049626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:11.106404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1126/2242] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:11 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:11.107897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726cdeb-a498-4fe5-b5b6-27edd1bdb3b3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.123943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b726cdeb-a498-4fe5-b5b6-27edd1bdb3b3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.292961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.293661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.293950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.294259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.294924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.299149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:11.340710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1117/2243] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:11 2022] POST /identity/v3/users => generated 299 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:11.343547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.357886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83826b8b-6871-4309-a329-265cb653952d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.366868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.367732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.370035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.370971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.372040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.377733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:11.392690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1127/2244] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:11.397787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.419242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16646597-e8ff-4346-b0b8-21baa6712bd0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.425835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.427438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:11.445681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8777e5ac-12c3-4bcb-8691-3830c678ca84 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.455526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.456517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.456802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.457079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] PATH_INFO: `/v3/projects/deee1a73e228459ea17b72dafe9fd738/users/6b280049322e4c51aa1fb890d496d700/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.459689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.460704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.461351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.462435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] PATH_INFO: `/v3/projects/be3af6e5ba244de3bb4f16570a4f42e3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.464429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=be3af6e5ba244de3bb4f16570a4f42e3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.466529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:11.492959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:11.527077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] RBAC: Authorizing `identity:create_grant(project_id=deee1a73e228459ea17b72dafe9fd738, user_id=6b280049322e4c51aa1fb890d496d700, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.543682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:11.588346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1118/2245] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:11 2022] PUT /identity/v3/projects/deee1a73e228459ea17b72dafe9fd738/users/6b280049322e4c51aa1fb890d496d700/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:11.589554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'be3af6e5ba244de3bb4f16570a4f42e3', 'request_id': 'req-e0671256-29f8-4d10-815b-aee4911084c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:11.589677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.600631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'be3af6e5ba244de3bb4f16570a4f42e3', 'request_id': 'req-e0671256-29f8-4d10-815b-aee4911084c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:11.612482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'be3af6e5ba244de3bb4f16570a4f42e3', 'request_id': 'req-e0671256-29f8-4d10-815b-aee4911084c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:11.637938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] The token cache is being invalidate because project be3af6e5ba244de3bb4f16570a4f42e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:11.638275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:11.638921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e0671256-29f8-4d10-815b-aee4911084c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:11.640731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1128/2246] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:11 2022] DELETE /identity/v3/projects/be3af6e5ba244de3bb4f16570a4f42e3 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:11.645844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24ba47-1c4c-4f97-9b4d-4817ba366b91 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.646141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24ba47-1c4c-4f97-9b4d-4817ba366b91 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.646577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c24ba47-1c4c-4f97-9b4d-4817ba366b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.671478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5c24ba47-1c4c-4f97-9b4d-4817ba366b91 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:11.673155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c24ba47-1c4c-4f97-9b4d-4817ba366b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:11.708872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0c90d1-27ab-4596-8129-a56a2680e37e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.716709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.717491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.718031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.718376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.719948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.726532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:11.743034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1119/2247] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:11.747329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.750900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:11.772980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c90672f-6427-42be-8781-8063db76c336 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.774641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1129/2248] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:11.779332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.780466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.781745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.782518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] PATH_INFO: `/v3/projects/deee1a73e228459ea17b72dafe9fd738/users/6b280049322e4c51aa1fb890d496d700/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.784306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578fc7-50e1-4954-89d7-915ee597f3d1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.784480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578fc7-50e1-4954-89d7-915ee597f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.784759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3578fc7-50e1-4954-89d7-915ee597f3d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.790807 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=deee1a73e228459ea17b72dafe9fd738, user_id=6b280049322e4c51aa1fb890d496d700, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.795768 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:11.815414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a3578fc7-50e1-4954-89d7-915ee597f3d1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:11.816060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1120/2249] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:11 2022] PUT /identity/v3/projects/deee1a73e228459ea17b72dafe9fd738/users/6b280049322e4c51aa1fb890d496d700/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:11.817601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a3578fc7-50e1-4954-89d7-915ee597f3d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:11.862392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:11.934181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1130/2250] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:11.935551 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:11.954737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fbcb90-c5d1-4d45-a077-21c7742279c4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:11.966817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:11.967541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:11.968085 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:11.968761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:11.971073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:11.971574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:11.981263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:12.011456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1121/2251] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:12.020052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578fc7-50e1-4954-89d7-915ee597f3d1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:12.044451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3578fc7-50e1-4954-89d7-915ee597f3d1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:12.065842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:12.066569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.066817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.067097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.068451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:12.072597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:12.134332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1131/2252] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:12 2022] POST /identity/v3/projects => generated 371 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:12.139111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:12.160637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56bf4a-1bdb-4dbc-9a1d-668a5722d10c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:12.324652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:12.326498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.328244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.328532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.334116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:12.340358 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:12.398630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1122/2253] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:12 2022] POST /identity/v3/users => generated 286 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:12.402656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:12.414783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53892242-06e4-4ca3-97d8-4edb247de906 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:12.433838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:12.434682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.435011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.435788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.436215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:12.438621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:12.459877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1132/2254] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:12.462938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:12.475038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddf7969-1c9c-4013-9b7d-2f6061de1131 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:12.481211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e22ac95-9568-4912-837c-51a17b84796c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:12.482191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ac95-9568-4912-837c-51a17b84796c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.482608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ac95-9568-4912-837c-51a17b84796c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.482982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e22ac95-9568-4912-837c-51a17b84796c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.484164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ac95-9568-4912-837c-51a17b84796c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:12.485824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e22ac95-9568-4912-837c-51a17b84796c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:12.498776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1123/2255] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:12.502735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:12.525210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55960bd-405f-4782-bb20-d97453fa5c3c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:12.530776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:12.531901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.532325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.532705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] PATH_INFO: `/v3/projects/3d3b58b3beed423aba9a7c28d1e09c88/users/f28b6a7ebf8c4cbab9193b95b8de6818/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.569767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d3b58b3beed423aba9a7c28d1e09c88, user_id=f28b6a7ebf8c4cbab9193b95b8de6818, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:12.586146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:12.632848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1133/2256] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:12 2022] PUT /identity/v3/projects/3d3b58b3beed423aba9a7c28d1e09c88/users/f28b6a7ebf8c4cbab9193b95b8de6818/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:12.635912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc9f1-b182-4f5d-ba98-e69f17f8ee12 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.636566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc9f1-b182-4f5d-ba98-e69f17f8ee12 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.637838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9bc9f1-b182-4f5d-ba98-e69f17f8ee12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.662524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bd9bc9f1-b182-4f5d-ba98-e69f17f8ee12 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:12.668606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bd9bc9f1-b182-4f5d-ba98-e69f17f8ee12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:12.702933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:12.746704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1124/2257] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:12.754326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:12.756144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:12.826742 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12d05f-5a07-4411-8bb3-dc4f71a1c924 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:12.830536 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333d876e-ca96-447b-8754-78b654277d59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:12.831220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d876e-ca96-447b-8754-78b654277d59 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.831550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d876e-ca96-447b-8754-78b654277d59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.831818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333d876e-ca96-447b-8754-78b654277d59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.848350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d876e-ca96-447b-8754-78b654277d59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:12.849954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333d876e-ca96-447b-8754-78b654277d59 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:12.861528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1134/2258] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:12.890380 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde83b74-e13e-4566-a954-7bb88c4fba3d None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.891621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde83b74-e13e-4566-a954-7bb88c4fba3d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.892313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde83b74-e13e-4566-a954-7bb88c4fba3d None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.894669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1125/2259] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:27:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:27:12.918192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:12.918490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:12.918775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:12.939587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] MFA Rules not processed for user `a0eae5ef8c1747408cbaad55be572987`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:12.941068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:13.064692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1135/2260] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:27:12 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:13.081793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde83b74-e13e-4566-a954-7bb88c4fba3d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:13.084574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fde83b74-e13e-4566-a954-7bb88c4fba3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:13.138651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde83b74-e13e-4566-a954-7bb88c4fba3d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:13.148183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:13.148552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:13.149445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:13.149741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:13.150000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] PATH_INFO: `/v3/users/730e7d9b92c84bd5872bc0e221b616cf` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:13.150664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:13.151942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=730e7d9b92c84bd5872bc0e221b616cf)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:13.153978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:13.179965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aaefccd-3f05-4ff0-829d-52a77870955f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:13.201586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:13.202431 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:13.202905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:13.203697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:13.220989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:13.227403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:13.233521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:13.268935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1136/2261] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:27:13.290461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '730e7d9b92c84bd5872bc0e221b616cf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.308994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '730e7d9b92c84bd5872bc0e221b616cf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.309854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '730e7d9b92c84bd5872bc0e221b616cf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.342494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '730e7d9b92c84bd5872bc0e221b616cf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.363104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7145931-0c10-4b2e-bd04-e35b0e50ec42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '730e7d9b92c84bd5872bc0e221b616cf'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.375694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1126/2262] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:13 2022] DELETE /identity/v3/users/730e7d9b92c84bd5872bc0e221b616cf => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:13.625825 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:13.643732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29046b3f-0187-4d5e-bb86-fdbeed505ad0 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:13.722754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:13.723635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:13.724069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:13.724353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] PATH_INFO: `/v3/projects/0c765bc215964bce8a321b4f3d085ae6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:13.725568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c765bc215964bce8a321b4f3d085ae6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:13.727502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:13.785992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c765bc215964bce8a321b4f3d085ae6', 'request_id': 'req-4fb358ab-951e-49ca-8810-0a0ec85baed7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.794427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c765bc215964bce8a321b4f3d085ae6', 'request_id': 'req-4fb358ab-951e-49ca-8810-0a0ec85baed7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.802665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c765bc215964bce8a321b4f3d085ae6', 'request_id': 'req-4fb358ab-951e-49ca-8810-0a0ec85baed7'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.825898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] The token cache is being invalidate because project 0c765bc215964bce8a321b4f3d085ae6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:13.826216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.826982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4fb358ab-951e-49ca-8810-0a0ec85baed7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:13.829132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1137/2263] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:13 2022] DELETE /identity/v3/projects/0c765bc215964bce8a321b4f3d085ae6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:13.834162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87fe0-368f-4f01-b05b-6cdbb05f1f84 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:13.834491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87fe0-368f-4f01-b05b-6cdbb05f1f84 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:13.834716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a87fe0-368f-4f01-b05b-6cdbb05f1f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:13.861226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b4a87fe0-368f-4f01-b05b-6cdbb05f1f84 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:13.862655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b4a87fe0-368f-4f01-b05b-6cdbb05f1f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:14.025837 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1127/2264] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:14.034200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cfb76-5f8e-4bf0-bfba-38afe03e9303 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.034503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cfb76-5f8e-4bf0-bfba-38afe03e9303 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.034789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9cfb76-5f8e-4bf0-bfba-38afe03e9303 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.061534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aa9cfb76-5f8e-4bf0-bfba-38afe03e9303 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:14.062708 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa9cfb76-5f8e-4bf0-bfba-38afe03e9303 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:14.089577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1138/2265] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:14.090945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87fe0-368f-4f01-b05b-6cdbb05f1f84 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:14.106658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a87fe0-368f-4f01-b05b-6cdbb05f1f84 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:14.120200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:14.120893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.121179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.121459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.122202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:14.122513 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:14.127330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:14.159615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1128/2266] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:14.164037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cfb76-5f8e-4bf0-bfba-38afe03e9303 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:14.180569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9cfb76-5f8e-4bf0-bfba-38afe03e9303 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:14.196195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0591e6cd-5f75-4db3-b3bf-85f379f5058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:14.198319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e6cd-5f75-4db3-b3bf-85f379f5058d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.198617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e6cd-5f75-4db3-b3bf-85f379f5058d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.200164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0591e6cd-5f75-4db3-b3bf-85f379f5058d admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.201380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591e6cd-5f75-4db3-b3bf-85f379f5058d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:14.203720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0591e6cd-5f75-4db3-b3bf-85f379f5058d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:14.255357 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1139/2267] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:14 2022] POST /identity/v3/projects => generated 355 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:14.263288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:14.277267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915f05ba-e87d-4c83-8a22-ac1f646389f8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:14.377179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52230b42-c397-4427-bd98-9212f08d477c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.377511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52230b42-c397-4427-bd98-9212f08d477c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.377760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52230b42-c397-4427-bd98-9212f08d477c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.378767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:14.419581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:14.420925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.421428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.422249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.423468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:14.427979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:14.443582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-52230b42-c397-4427-bd98-9212f08d477c None None] MFA Rules not processed for user `f28b6a7ebf8c4cbab9193b95b8de6818`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:14.448285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-52230b42-c397-4427-bd98-9212f08d477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:14.460782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1129/2268] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:14 2022] POST /identity/v3/users => generated 278 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:14.466117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:14.507638 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbc9d22-81ad-4d01-8cbf-def73f3b4540 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:14.529138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:14.529984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.530314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.535471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.535471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:14.555531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:14.572730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:14.600751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1130/2269] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:14.601849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:14.626593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440ef688-b5c3-431b-a113-5af7af891dcf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:14.627458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1140/2270] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:14.643400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52230b42-c397-4427-bd98-9212f08d477c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:14.646423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-52230b42-c397-4427-bd98-9212f08d477c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:14.652008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:14.659254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.659549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.659871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.660654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:14.668010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:14.703062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1131/2271] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:14 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:14.704015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:14.715330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f241469-3895-43ac-9417-f988fceefe65 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:14.723895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409a4fcd-d677-4f31-983c-439026683589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:14.724870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a4fcd-d677-4f31-983c-439026683589 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.725192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a4fcd-d677-4f31-983c-439026683589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.725433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409a4fcd-d677-4f31-983c-439026683589 admin admin] PATH_INFO: `/v3/projects/77014643111f44f88c6737c06c9188a4/users/0978c4e007ee44f9b2c9495ed9fae6c7/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.757931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a4fcd-d677-4f31-983c-439026683589 admin admin] RBAC: Authorizing `identity:create_grant(project_id=77014643111f44f88c6737c06c9188a4, user_id=0978c4e007ee44f9b2c9495ed9fae6c7, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:14.761732 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409a4fcd-d677-4f31-983c-439026683589 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:14.788644 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1132/2272] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:14 2022] PUT /identity/v3/projects/77014643111f44f88c6737c06c9188a4/users/0978c4e007ee44f9b2c9495ed9fae6c7/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:14.798821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52230b42-c397-4427-bd98-9212f08d477c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:14.800806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2568-3a5d-4768-8bc5-20be5fe50984 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.801098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2568-3a5d-4768-8bc5-20be5fe50984 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.801489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbe2568-3a5d-4768-8bc5-20be5fe50984 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.848847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2cbe2568-3a5d-4768-8bc5-20be5fe50984 None None] MFA Rules not processed for user `0978c4e007ee44f9b2c9495ed9fae6c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:14.857281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2cbe2568-3a5d-4768-8bc5-20be5fe50984 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:14.903892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:14.907135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:14.907903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:14.909001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:14.983257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1133/2273] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:14 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:14.987654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2568-3a5d-4768-8bc5-20be5fe50984 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:15.036649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:15.052584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:15.087186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1141/2274] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:15.125778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbe2568-3a5d-4768-8bc5-20be5fe50984 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:15.138246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:15.145457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:15.145873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:15.146140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:15.147623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:15.154852 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:15.194485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:15.225018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1134/2275] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:14 2022] POST /identity/v3/projects => generated 355 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:15.229967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:15.247083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5770d3a2-4bf1-48cf-9e75-0ba6f8c78b5d service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:15.478539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:15.503140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:15.504602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:15.506273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:15.507336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:15.508615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:15.520789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:15.588351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1142/2276] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:15 2022] POST /identity/v3/users => generated 284 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:15.589285 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:15.606527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35032924-5b52-4192-b7bb-67b8e266fafc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:15.617083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:15.617943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:15.622079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:15.622697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:15.625559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:15.631242 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:15.654343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1135/2277] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:15.659614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:15.677636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776392a8-72aa-457a-a692-1aee00ff72f9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:15.698997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:15.700299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:15.700853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:15.701320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] PATH_INFO: `/v3/projects/1372ac0621594eeb9dceee411e81c9f6/users/422de4ee7e534a8f80705c4ac3e640fd/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:15.746862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1372ac0621594eeb9dceee411e81c9f6, user_id=422de4ee7e534a8f80705c4ac3e640fd, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:15.751360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:15.775783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1143/2278] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:15 2022] PUT /identity/v3/projects/1372ac0621594eeb9dceee411e81c9f6/users/422de4ee7e534a8f80705c4ac3e640fd/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:15.782526 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:15.796194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef585d3f-550e-4ce5-90fe-002ce206118a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:15.955435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00a44ac-938d-4f4d-9833-b67b8ab47554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:15.960782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a44ac-938d-4f4d-9833-b67b8ab47554 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:15.961190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a44ac-938d-4f4d-9833-b67b8ab47554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:15.961554 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00a44ac-938d-4f4d-9833-b67b8ab47554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:15.964129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a44ac-938d-4f4d-9833-b67b8ab47554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:15.967238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00a44ac-938d-4f4d-9833-b67b8ab47554 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:15.994205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1136/2279] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:15 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:15.999410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:16.021376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a9ac60-f835-486f-8c41-4c9c21e885be admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:16.034712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:16.037013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:16.037859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:16.037859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] PATH_INFO: `/v3/projects/1372ac0621594eeb9dceee411e81c9f6/users/422de4ee7e534a8f80705c4ac3e640fd/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:16.040083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1372ac0621594eeb9dceee411e81c9f6, user_id=422de4ee7e534a8f80705c4ac3e640fd, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:16.043977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:16.062466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1144/2280] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:15 2022] PUT /identity/v3/projects/1372ac0621594eeb9dceee411e81c9f6/users/422de4ee7e534a8f80705c4ac3e640fd/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:16.073388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:16.074010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:16.074631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:16.128762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] MFA Rules not processed for user `422de4ee7e534a8f80705c4ac3e640fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:16.130803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:16.276288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1137/2281] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:16 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:16.291176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:16.437889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b5fda-ed69-45b8-a0d8-241e9ff34b46 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:16.442015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3290ea1-735f-42c8-aadf-f5700c91ed7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:16.442689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3290ea1-735f-42c8-aadf-f5700c91ed7d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:16.442974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3290ea1-735f-42c8-aadf-f5700c91ed7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:16.443256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3290ea1-735f-42c8-aadf-f5700c91ed7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:16.459006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3290ea1-735f-42c8-aadf-f5700c91ed7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:16.460631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3290ea1-735f-42c8-aadf-f5700c91ed7d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:16.472705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1145/2282] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:16 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:16.590406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:16.597174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:16.619300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f423d-fae2-4e9e-b345-271c89778725 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:16.623435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:16.624074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:16.624357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:16.624634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:16.740211 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:16.742680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:16.755615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1138/2283] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:16 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:17.072039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d88ec-21f2-4e9e-abdf-521654aeaaa0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:17.073984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d88ec-21f2-4e9e-abdf-521654aeaaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:17.074485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6d88ec-21f2-4e9e-abdf-521654aeaaa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:17.116943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7a6d88ec-21f2-4e9e-abdf-521654aeaaa0 None None] MFA Rules not processed for user `f28b6a7ebf8c4cbab9193b95b8de6818`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:17.118524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7a6d88ec-21f2-4e9e-abdf-521654aeaaa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:17.264364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1146/2284] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:17.605103 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:17.610199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:17.701848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e7789-062e-4342-9720-ba02af01478c service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:17.707641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:17.708965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:17.709552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:17.710113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] PATH_INFO: `/v3/users/0978c4e007ee44f9b2c9495ed9fae6c7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:17.711523 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0978c4e007ee44f9b2c9495ed9fae6c7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:17.713903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:17.776998 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0978c4e007ee44f9b2c9495ed9fae6c7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:17.800339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0978c4e007ee44f9b2c9495ed9fae6c7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:17.802931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0978c4e007ee44f9b2c9495ed9fae6c7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:17.811836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0978c4e007ee44f9b2c9495ed9fae6c7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:17.837015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0978c4e007ee44f9b2c9495ed9fae6c7'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:17.851327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1139/2285] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:17 2022] DELETE /identity/v3/users/0978c4e007ee44f9b2c9495ed9fae6c7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:17.856775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:17.857178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:17.857688 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:17.885837 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:17.887886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:17.970741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:17.987004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1147/2286] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:17.997189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:17.999272 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:18.089841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1770acf7-8170-4557-9932-bb7d22d4b42f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:18.100636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:18.100636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:18.100923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:18.101190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:18.114344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:18.116028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:18.136889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1140/2287] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:18.657701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:18.659712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:18.691540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae865f68-44c1-42f3-82b0-7850b454ac91 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:18.697526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:18.699988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:18.700486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:18.701030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] PATH_INFO: `/v3/projects/77014643111f44f88c6737c06c9188a4` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:18.702584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=77014643111f44f88c6737c06c9188a4)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:18.704774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:18.783238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77014643111f44f88c6737c06c9188a4', 'request_id': 'req-5a4688a4-ff6f-461a-a27d-0e734184fd82'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:18.796027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77014643111f44f88c6737c06c9188a4', 'request_id': 'req-5a4688a4-ff6f-461a-a27d-0e734184fd82'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:18.838195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77014643111f44f88c6737c06c9188a4', 'request_id': 'req-5a4688a4-ff6f-461a-a27d-0e734184fd82'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:18.859116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] The token cache is being invalidate because project 77014643111f44f88c6737c06c9188a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:18.859544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:18.860350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:18.862467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1148/2288] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:18 2022] DELETE /identity/v3/projects/77014643111f44f88c6737c06c9188a4 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:18.866427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:18.882521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e87d692-2404-42e8-bf8a-ff37d49ca58c service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:18.935410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:19.047937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:19.050150 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:19.050445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:19.050729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] PATH_INFO: `/v3/users/422de4ee7e534a8f80705c4ac3e640fd` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:19.055144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] RBAC: Authorizing `identity:delete_user(user_id=422de4ee7e534a8f80705c4ac3e640fd)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:19.057770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:19.205171 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '422de4ee7e534a8f80705c4ac3e640fd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:19.233194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '422de4ee7e534a8f80705c4ac3e640fd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:19.234218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '422de4ee7e534a8f80705c4ac3e640fd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:19.252418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '422de4ee7e534a8f80705c4ac3e640fd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:19.295649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-a25a622e-a6e2-43d7-b0d8-f1581feb316f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '422de4ee7e534a8f80705c4ac3e640fd'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:19.310381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1141/2289] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:18 2022] DELETE /identity/v3/users/422de4ee7e534a8f80705c4ac3e640fd => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:19.841996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:19.844071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:19.875781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4688a4-ff6f-461a-a27d-0e734184fd82 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:20.051293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:20.052482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.052913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.053363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] PATH_INFO: `/v3/projects/1372ac0621594eeb9dceee411e81c9f6` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.074200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1372ac0621594eeb9dceee411e81c9f6)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:20.077297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:20.218973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1372ac0621594eeb9dceee411e81c9f6', 'request_id': 'req-4d6b8450-45c2-4d78-a1b6-68aec86821e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.238331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1372ac0621594eeb9dceee411e81c9f6', 'request_id': 'req-4d6b8450-45c2-4d78-a1b6-68aec86821e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.252680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1372ac0621594eeb9dceee411e81c9f6', 'request_id': 'req-4d6b8450-45c2-4d78-a1b6-68aec86821e5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.287335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] The token cache is being invalidate because project 1372ac0621594eeb9dceee411e81c9f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:20.287962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.295718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d6b8450-45c2-4d78-a1b6-68aec86821e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.301754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1149/2290] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:19 2022] DELETE /identity/v3/projects/1372ac0621594eeb9dceee411e81c9f6 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:20.312177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb266c-a839-48cd-85e3-ab85e0b02cfe None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.312896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb266c-a839-48cd-85e3-ab85e0b02cfe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.313273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb266c-a839-48cd-85e3-ab85e0b02cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.345421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4fdb266c-a839-48cd-85e3-ab85e0b02cfe None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:20.349864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4fdb266c-a839-48cd-85e3-ab85e0b02cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:20.489694 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.490061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.490453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.543855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1142/2291] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:20.550646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ed22-971b-403d-a3e2-92758f29691d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.551172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ed22-971b-403d-a3e2-92758f29691d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.551597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce22ed22-971b-403d-a3e2-92758f29691d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.555137 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:20.570752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] MFA Rules not processed for user `6b280049322e4c51aa1fb890d496d700`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:20.575597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:20.597654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ce22ed22-971b-403d-a3e2-92758f29691d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:20.598933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ce22ed22-971b-403d-a3e2-92758f29691d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:20.615793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1143/2292] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:20.621951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ed22-971b-403d-a3e2-92758f29691d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:20.646158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce22ed22-971b-403d-a3e2-92758f29691d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:20.663596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:20.666410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.666786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.667504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.671467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:20.672492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:20.675179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:20.703081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1144/2293] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:20.707420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:20.743876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1150/2294] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:20 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:20.745041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:20.746689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:20.747955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef8b676-c92a-44dd-baa9-c9878c39f811 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:20.753799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:20.754646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.755074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.755495 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.756386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:20.759531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:20.777945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ff48b1-5af3-4667-a815-2dfa04b6e457 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:20.789820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:20.793230 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.793755 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.794243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] PATH_INFO: `/v3/users/f66cb5f2ec024a1f97e1c2fa47cc2dc1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.795813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=f66cb5f2ec024a1f97e1c2fa47cc2dc1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:20.798109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:20.806170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1145/2295] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:20 2022] POST /identity/v3/projects => generated 387 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:20.807334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:20.811573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:20.818440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:20.914268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f66cb5f2ec024a1f97e1c2fa47cc2dc1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.922135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2157e39-c17b-4d28-8acf-7a1e455256e9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:20.925274 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f66cb5f2ec024a1f97e1c2fa47cc2dc1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.926252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f66cb5f2ec024a1f97e1c2fa47cc2dc1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.927070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ad630-19bc-4bb4-9c11-abc06cb95e09 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:20.927722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ad630-19bc-4bb4-9c11-abc06cb95e09 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:20.928020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ad630-19bc-4bb4-9c11-abc06cb95e09 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:20.928289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ad630-19bc-4bb4-9c11-abc06cb95e09 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:20.930451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-216ad630-19bc-4bb4-9c11-abc06cb95e09 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:20.942738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f66cb5f2ec024a1f97e1c2fa47cc2dc1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.953826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f66cb5f2ec024a1f97e1c2fa47cc2dc1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:20.960726 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1151/2296] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:20 2022] DELETE /identity/v3/users/f66cb5f2ec024a1f97e1c2fa47cc2dc1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:20.966916 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:20.968483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.019333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ad630-19bc-4bb4-9c11-abc06cb95e09 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.021089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ad630-19bc-4bb4-9c11-abc06cb95e09 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.033006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1146/2297] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:21.034949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.035342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.036125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.064348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff0edc-8896-4611-9b48-5f8537e143ca admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.074948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:21.075150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.075547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.075920 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.076215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.076316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.078716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.082565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.089480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1147/2298] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:21.100591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.102366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.105679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:21.120542 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1152/2299] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:20 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:21.122242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.138288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd569419-ff0f-4b0b-a010-47d2cdd6aadc admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.145103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.146011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.146368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.146759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.148040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.151427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.172142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1153/2300] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:21.173165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.186855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eff37b2-9704-4f93-aac2-81c938db7485 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.195569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.196503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.197342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.197682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.198530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.201807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.218186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1154/2301] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:21.222484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.230101 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb26f11a-9554-43f6-b4b9-14895b33bc06 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.235379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e02b16-f7c7-4135-8531-53bfb17cbbb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.236122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e02b16-f7c7-4135-8531-53bfb17cbbb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.236731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e02b16-f7c7-4135-8531-53bfb17cbbb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.236971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e02b16-f7c7-4135-8531-53bfb17cbbb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.238615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7d55b3-4a17-4481-a44e-7fec9bf16e3c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.248449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdeb1ce-96d1-40c4-b6fc-efa99e346dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.249378 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeb1ce-96d1-40c4-b6fc-efa99e346dea admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.249663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeb1ce-96d1-40c4-b6fc-efa99e346dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.249947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdeb1ce-96d1-40c4-b6fc-efa99e346dea admin admin] PATH_INFO: `/v3/projects/1c7909f48262461eaf8ede4f9f89e56b/users/c96f8d51597849af845808d302597da4/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.252232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e02b16-f7c7-4135-8531-53bfb17cbbb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.253746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e02b16-f7c7-4135-8531-53bfb17cbbb9 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.264667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1148/2302] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:27:21.265872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.266163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.266449 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.274056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeb1ce-96d1-40c4-b6fc-efa99e346dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c7909f48262461eaf8ede4f9f89e56b, user_id=c96f8d51597849af845808d302597da4, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.277902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdeb1ce-96d1-40c4-b6fc-efa99e346dea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.292065 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:21.295502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.301643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1155/2303] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:21 2022] PUT /identity/v3/projects/1c7909f48262461eaf8ede4f9f89e56b/users/c96f8d51597849af845808d302597da4/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:21.312689 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eed48-d6ca-40ac-8d9a-1f866717fe04 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.312992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eed48-d6ca-40ac-8d9a-1f866717fe04 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.313343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98eed48-d6ca-40ac-8d9a-1f866717fe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.350234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d98eed48-d6ca-40ac-8d9a-1f866717fe04 None None] MFA Rules not processed for user `c96f8d51597849af845808d302597da4`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:21.352077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d98eed48-d6ca-40ac-8d9a-1f866717fe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.355049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:21.373781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1149/2304] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:21.419803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.421307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.439580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1156/2305] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:21.444603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98eed48-d6ca-40ac-8d9a-1f866717fe04 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.512997 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b9d41f-fd6d-4471-b6ec-5d4ef9512140 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.526040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98eed48-d6ca-40ac-8d9a-1f866717fe04 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.527001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.527825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.528114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.528398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.531972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.532672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.533018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.533298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.534087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.536424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.539167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.541684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.563919 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1150/2306] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:27:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 07 10:27:21.593294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1157/2307] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:21 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:27:21.595573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.609750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bbc6c2-88a2-43f0-a266-5460f370adc1 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.614607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.620786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.680224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:21.729154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.729945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.730395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.730755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.731196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f2258-1449-49a4-8b99-74155b6aba87 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.731955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.734199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.736949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.737828 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.738106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.738375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] PATH_INFO: `/v3/projects/9f1d5f362a4d410d846ddf2e919da8cb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.739812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f1d5f362a4d410d846ddf2e919da8cb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.741854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.780444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1151/2308] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:21 2022] POST /identity/v3/users => generated 299 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:21.784012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:21.805668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786f503-fbaf-4412-942b-068ed4ab1bfa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:21.825770 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f1d5f362a4d410d846ddf2e919da8cb', 'request_id': 'req-15938a3c-5ca3-4731-98a8-8390735be8b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:21.845361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f1d5f362a4d410d846ddf2e919da8cb', 'request_id': 'req-15938a3c-5ca3-4731-98a8-8390735be8b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:21.856568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f1d5f362a4d410d846ddf2e919da8cb', 'request_id': 'req-15938a3c-5ca3-4731-98a8-8390735be8b5'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:21.872262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] The token cache is being invalidate because project 9f1d5f362a4d410d846ddf2e919da8cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:21.872785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:21.886588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-15938a3c-5ca3-4731-98a8-8390735be8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:21.897245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1158/2309] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:21 2022] DELETE /identity/v3/projects/9f1d5f362a4d410d846ddf2e919da8cb => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:21.903003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272d43d-f80c-40cf-93c5-d4c13c2e41d6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.903430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272d43d-f80c-40cf-93c5-d4c13c2e41d6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.903730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272d43d-f80c-40cf-93c5-d4c13c2e41d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.957197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b272d43d-f80c-40cf-93c5-d4c13c2e41d6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:21.962522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b272d43d-f80c-40cf-93c5-d4c13c2e41d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:21.965584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:21.966636 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:21.967126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:21.967734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:21.968921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:21.970747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:21.986090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1159/2310] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:22.000400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8435d-718d-48fb-bff1-b0fc1fb384b0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.000982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8435d-718d-48fb-bff1-b0fc1fb384b0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.001458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab8435d-718d-48fb-bff1-b0fc1fb384b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.009741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1152/2311] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:22.011505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.015170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:22.057043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4ab8435d-718d-48fb-bff1-b0fc1fb384b0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:22.057280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b222b43-b22e-4284-a353-6277018c7612 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.058491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4ab8435d-718d-48fb-bff1-b0fc1fb384b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:22.074157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.075208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.075612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.076004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] PATH_INFO: `/v3/projects/d71883b5a6bb47709a1a9adccb127b42/users/98a2c3c99ce4434bbd3c03478108ff12/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.083196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1160/2312] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:22.088301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8435d-718d-48fb-bff1-b0fc1fb384b0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.093898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:22.105525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71883b5a6bb47709a1a9adccb127b42, user_id=98a2c3c99ce4434bbd3c03478108ff12, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.108606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab8435d-718d-48fb-bff1-b0fc1fb384b0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.115909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.117955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.119631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.119966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.120237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.121910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:22.122123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.124318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.146398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1161/2313] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:22.147924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.160368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1153/2314] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:22 2022] PUT /identity/v3/projects/d71883b5a6bb47709a1a9adccb127b42/users/98a2c3c99ce4434bbd3c03478108ff12/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:22.161320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.175241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff075f2-667b-48ec-9847-caff80ce874f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.257857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.258540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.258886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.259329 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.260061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.262033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.276235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239e8fd-55de-45bf-a42d-4685678fa235 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.284967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.285786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.286077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.286359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.287217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.294470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.302250 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1162/2315] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:22 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:22.307111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.312836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1154/2316] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:22.319026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.332811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dbb3da-46ce-4b28-a5a2-6dfe3342d010 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.338473 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:22.409672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa61bc5-e189-4528-8e5a-30ee7ddeb67c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.410615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61bc5-e189-4528-8e5a-30ee7ddeb67c admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.410907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61bc5-e189-4528-8e5a-30ee7ddeb67c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.411188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa61bc5-e189-4528-8e5a-30ee7ddeb67c admin admin] PATH_INFO: `/v3/projects/d71883b5a6bb47709a1a9adccb127b42/users/98a2c3c99ce4434bbd3c03478108ff12/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.411644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b144c6-efe5-4f65-9b5a-4b132100dfd8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.414870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa61bc5-e189-4528-8e5a-30ee7ddeb67c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d71883b5a6bb47709a1a9adccb127b42, user_id=98a2c3c99ce4434bbd3c03478108ff12, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.418192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa61bc5-e189-4528-8e5a-30ee7ddeb67c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.418761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.419629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.420027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.420413 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.421535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.424563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.430061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1155/2317] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:22 2022] PUT /identity/v3/projects/d71883b5a6bb47709a1a9adccb127b42/users/98a2c3c99ce4434bbd3c03478108ff12/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:22.441462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.441755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.442658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.451918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1163/2318] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:22 2022] POST /identity/v3/users => generated 287 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:22.454520 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.469682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7e3ed-7c41-4c1d-b594-ff31b8977361 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.485337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] MFA Rules not processed for user `98a2c3c99ce4434bbd3c03478108ff12`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:22.486969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:22.625305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.626012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.626311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.626588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.627464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.629256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.644375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1164/2319] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:22.648389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.664543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa661b31-9684-4ce2-a8ec-62802ace7b65 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.667153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1156/2320] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:22.671736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.673513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.674003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.674475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.675418 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.677478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.688252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.689749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1165/2321] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:22 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:22.689882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:22.693383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.707311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d1c31-413d-4ba3-9806-26d2d9541193 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.713628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8ac8-b539-490a-93db-c6e17b5dd3fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.714722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8ac8-b539-490a-93db-c6e17b5dd3fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.715166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8ac8-b539-490a-93db-c6e17b5dd3fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.715607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8ac8-b539-490a-93db-c6e17b5dd3fd admin admin] PATH_INFO: `/v3/projects/2ef29d7b1c33443087a1d90409dd932c/users/bbe6b7c6169d48aea6253c308236e4b3/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.726173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:22.755319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8ac8-b539-490a-93db-c6e17b5dd3fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef29d7b1c33443087a1d90409dd932c, user_id=bbe6b7c6169d48aea6253c308236e4b3, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:22.758139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8ac8-b539-490a-93db-c6e17b5dd3fd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:22.792410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1166/2322] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:22 2022] PUT /identity/v3/projects/2ef29d7b1c33443087a1d90409dd932c/users/bbe6b7c6169d48aea6253c308236e4b3/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:22.794336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.794682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.795034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.805058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc2336d-534e-4421-bd8d-9dfa1185f63a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.829447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] MFA Rules not processed for user `bbe6b7c6169d48aea6253c308236e4b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:22.831541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:22.887547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.889132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.889423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.889701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:22.930631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:22.949926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1167/2323] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:22.967463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:22.971128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:22.984609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977819bb-0849-452b-bdb1-c7541dbbc411 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:22.989225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920165c-3efa-4667-ad46-f1d61135cd42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:22.989998 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920165c-3efa-4667-ad46-f1d61135cd42 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:22.990539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920165c-3efa-4667-ad46-f1d61135cd42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:22.991252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920165c-3efa-4667-ad46-f1d61135cd42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:23.015924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920165c-3efa-4667-ad46-f1d61135cd42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:23.019185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920165c-3efa-4667-ad46-f1d61135cd42 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:23.021595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:23.023220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:23.044274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1157/2324] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:23.080434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1168/2325] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:23.101275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:23.305426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9192bcc-d0e3-428d-b4b8-f8e055336b87 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:23.311629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:23.313619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:23.313911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:23.314197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] PATH_INFO: `/v3/users/c96f8d51597849af845808d302597da4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:23.315991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96f8d51597849af845808d302597da4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:23.318188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:23.374679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:23.495218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96f8d51597849af845808d302597da4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:23.529882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96f8d51597849af845808d302597da4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:23.530821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96f8d51597849af845808d302597da4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:23.550676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96f8d51597849af845808d302597da4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:23.582130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96f8d51597849af845808d302597da4'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:23.597474 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1158/2326] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:23 2022] DELETE /identity/v3/users/c96f8d51597849af845808d302597da4 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:23.599275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:23.599727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:23.600183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:23.643890 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:23.648619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:23.822665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1169/2327] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:23.830001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:23.832667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:23.979568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b02b461-b9cb-4520-881b-37dd49020040 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:23.993799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:23.993799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:23.993799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:23.993799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:24.027319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:24.028981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:24.060323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1159/2328] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:24.672163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:24.673771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:24.700038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264352d1-2f6d-49cd-8570-fd7e22052fc3 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:24.705905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:24.707169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:24.707898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:24.708397 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] PATH_INFO: `/v3/projects/1c7909f48262461eaf8ede4f9f89e56b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:24.709848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c7909f48262461eaf8ede4f9f89e56b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:24.712069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:24.833026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c7909f48262461eaf8ede4f9f89e56b', 'request_id': 'req-381ec22b-df69-4772-8f56-84db80f368b0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:24.852283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c7909f48262461eaf8ede4f9f89e56b', 'request_id': 'req-381ec22b-df69-4772-8f56-84db80f368b0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:24.861991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c7909f48262461eaf8ede4f9f89e56b', 'request_id': 'req-381ec22b-df69-4772-8f56-84db80f368b0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:24.895629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] The token cache is being invalidate because project 1c7909f48262461eaf8ede4f9f89e56b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:24.896879 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:24.899526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:24.904421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1170/2329] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:24 2022] DELETE /identity/v3/projects/1c7909f48262461eaf8ede4f9f89e56b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:24.906156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:24.932370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6b10f-1892-4ebc-9ac7-b137eb022745 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:25.043480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:25.044438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:25.044722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:25.044999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] PATH_INFO: `/v3/users/98a2c3c99ce4434bbd3c03478108ff12` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:25.046676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a2c3c99ce4434bbd3c03478108ff12)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:25.048968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:25.112451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:25.132806 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a2c3c99ce4434bbd3c03478108ff12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:25.144408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a2c3c99ce4434bbd3c03478108ff12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:25.145332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a2c3c99ce4434bbd3c03478108ff12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:25.160574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a2c3c99ce4434bbd3c03478108ff12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:25.177426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-7f5cf3e2-cb47-4657-81e1-1374f144d380 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a2c3c99ce4434bbd3c03478108ff12'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:25.189135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1160/2330] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:24 2022] DELETE /identity/v3/users/98a2c3c99ce4434bbd3c03478108ff12 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:25.754892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:25.756422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:25.787165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381ec22b-df69-4772-8f56-84db80f368b0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:25.899104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:25.900270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:25.900612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:25.900992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] PATH_INFO: `/v3/projects/d71883b5a6bb47709a1a9adccb127b42` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:25.902392 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d71883b5a6bb47709a1a9adccb127b42)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:25.905213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:25.921529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:26.015729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd71883b5a6bb47709a1a9adccb127b42', 'request_id': 'req-e2afa90a-5d2f-4c3a-92c9-4075de977169'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:26.044312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd71883b5a6bb47709a1a9adccb127b42', 'request_id': 'req-e2afa90a-5d2f-4c3a-92c9-4075de977169'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:26.063136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd71883b5a6bb47709a1a9adccb127b42', 'request_id': 'req-e2afa90a-5d2f-4c3a-92c9-4075de977169'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:26.088610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] The token cache is being invalidate because project d71883b5a6bb47709a1a9adccb127b42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:26.088902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:26.090921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e2afa90a-5d2f-4c3a-92c9-4075de977169 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:26.093735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1171/2331] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:25 2022] DELETE /identity/v3/projects/d71883b5a6bb47709a1a9adccb127b42 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:26.098652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360672b8-1850-4036-a832-a7908a3c6217 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.099185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360672b8-1850-4036-a832-a7908a3c6217 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.099861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360672b8-1850-4036-a832-a7908a3c6217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.131711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-360672b8-1850-4036-a832-a7908a3c6217 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:26.133213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-360672b8-1850-4036-a832-a7908a3c6217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:26.226337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1161/2332] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:26.234320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f7a90-457e-427c-a4dd-86195bf44c79 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.234782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f7a90-457e-427c-a4dd-86195bf44c79 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.235166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f7a90-457e-427c-a4dd-86195bf44c79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.260243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-500f7a90-457e-427c-a4dd-86195bf44c79 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:26.264228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-500f7a90-457e-427c-a4dd-86195bf44c79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:26.288445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1172/2333] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:26.292968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360672b8-1850-4036-a832-a7908a3c6217 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:26.307463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360672b8-1850-4036-a832-a7908a3c6217 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:26.327394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:26.328338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.328922 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.330712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.331839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:26.333238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:26.335209 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:26.362006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1162/2334] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:27:26.366407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f7a90-457e-427c-a4dd-86195bf44c79 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:26.384390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f7a90-457e-427c-a4dd-86195bf44c79 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:26.395812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:26.397124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.397769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.398058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.398931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:26.402340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:26.500903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1173/2335] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:26 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:26.507186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:26.531220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644258d-6b3a-48fb-b820-23dbab8736ca admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:26.637048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:26.637738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.638021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.638300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.639000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:26.641932 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:26.668466 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1163/2336] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:26 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:26.672094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:26.684061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d45e87-11d9-4ba7-aadc-fc64709e9019 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:26.697901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:26.697901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.697901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.697901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.698468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:26.700338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:26.714369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1174/2337] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:26.718390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:26.739470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583ccf3-5285-4ade-a3f3-93312e554bd2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:26.752518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f845e04-b981-4688-84d9-790d0faa0778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:26.753981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f845e04-b981-4688-84d9-790d0faa0778 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.754948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f845e04-b981-4688-84d9-790d0faa0778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.755265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f845e04-b981-4688-84d9-790d0faa0778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.756876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f845e04-b981-4688-84d9-790d0faa0778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:26.759210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f845e04-b981-4688-84d9-790d0faa0778 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:26.781937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1164/2338] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:26.783046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:26.794697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564c833e-55f8-46f8-8b55-ede983314424 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:26.809177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:26.810144 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.810425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.810666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] PATH_INFO: `/v3/projects/b6431cf402df4f76ba422a5b8f9fea4b/users/2f5f96917087414fa11317541ef7ef13/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.850025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6431cf402df4f76ba422a5b8f9fea4b, user_id=2f5f96917087414fa11317541ef7ef13, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:26.855510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:26.887612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1175/2339] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:26 2022] PUT /identity/v3/projects/b6431cf402df4f76ba422a5b8f9fea4b/users/2f5f96917087414fa11317541ef7ef13/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:26.897507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581ca33-07c8-4904-9931-4877cf52daa3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:26.897795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581ca33-07c8-4904-9931-4877cf52daa3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:26.898142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f581ca33-07c8-4904-9931-4877cf52daa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:26.955444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f581ca33-07c8-4904-9931-4877cf52daa3 None None] MFA Rules not processed for user `2f5f96917087414fa11317541ef7ef13`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:26.957067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f581ca33-07c8-4904-9931-4877cf52daa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:26.964194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:27.091408 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1165/2340] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:27.095643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:27.226625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0819635e-cbda-40b0-a195-bd7344742840 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:27.232918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:27.233614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:27.233911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:27.234512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:27.235231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:27.237204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:27.272560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1176/2341] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:27 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:27.277745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581ca33-07c8-4904-9931-4877cf52daa3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:27.293236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f581ca33-07c8-4904-9931-4877cf52daa3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:27.437039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:27.438941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:27.440177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:27.440582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:27.441424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:27.443489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:27.474776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1166/2342] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:27 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:27.478989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:27.498653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd9a97-6786-467e-b981-13aca4d32432 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:27.513105 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:27.513819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:27.514128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:27.514411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:27.515160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:27.516808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:27.533841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1177/2343] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:27.537753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:27.566400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf06ba8-bf50-4aff-9ac6-667498b8d15b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:27.585886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e4e5d0-74cc-4e62-b3ae-6273e2eb3f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:27.589392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e5d0-74cc-4e62-b3ae-6273e2eb3f32 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:27.590928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e5d0-74cc-4e62-b3ae-6273e2eb3f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:27.591307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e4e5d0-74cc-4e62-b3ae-6273e2eb3f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:27.593917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4e5d0-74cc-4e62-b3ae-6273e2eb3f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:27.600158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e4e5d0-74cc-4e62-b3ae-6273e2eb3f32 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:27.626746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1167/2344] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:27 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:27.630893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:27.685669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212ca74b-bcf2-418e-9041-05f69ea08ae7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:27.693607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:27.694452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:27.694743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:27.695280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] PATH_INFO: `/v3/projects/4f24942bdfd04c77a8df36a19bd3e820/users/5e3c9034c4ab4d548d8f0ea26094b28b/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:27.698222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:27.716008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f24942bdfd04c77a8df36a19bd3e820, user_id=5e3c9034c4ab4d548d8f0ea26094b28b, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:27.718936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:27.761332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8a072-ae5f-4031-9984-d7b53270ef6e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:27.761696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8a072-ae5f-4031-9984-d7b53270ef6e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:27.761899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f8a072-ae5f-4031-9984-d7b53270ef6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:27.776163 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1178/2345] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:27 2022] PUT /identity/v3/projects/4f24942bdfd04c77a8df36a19bd3e820/users/5e3c9034c4ab4d548d8f0ea26094b28b/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:27.829814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-90f8a072-ae5f-4031-9984-d7b53270ef6e None None] MFA Rules not processed for user `5e3c9034c4ab4d548d8f0ea26094b28b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:27.832869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-90f8a072-ae5f-4031-9984-d7b53270ef6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:27.953956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1168/2346] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:27.959183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:28.063321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17987c0c-a563-4895-95bd-cb43f69f2080 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:28.073619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:28.074309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.074599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.074963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.075680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:28.077921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:28.143539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1179/2347] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:27 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:28.148026 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8a072-ae5f-4031-9984-d7b53270ef6e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:28.168761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f8a072-ae5f-4031-9984-d7b53270ef6e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:28.201602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:28.271786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:28.272780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.273166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.273443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.274149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:28.276460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:28.311844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1169/2348] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:28 2022] POST /identity/v3/users => generated 299 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:28.316675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:28.330984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0246eb8-9f89-447d-96d7-523afb7556f0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:28.338801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:28.339614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.339933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.340305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.341179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:28.343213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:28.358894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1180/2349] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:28.363006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:28.378149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c013d48d-48b1-4d14-9e4f-e5011c5ad760 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:28.389290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:28.390380 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.390762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.391110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] PATH_INFO: `/v3/projects/e0f8035f7863430ca4c37beec257ed53/users/127e0f8eee4d4a3eb0d5680fd4b5040a/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.420490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f8035f7863430ca4c37beec257ed53, user_id=127e0f8eee4d4a3eb0d5680fd4b5040a, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:28.425006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:28.450292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1170/2350] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:28 2022] PUT /identity/v3/projects/e0f8035f7863430ca4c37beec257ed53/users/127e0f8eee4d4a3eb0d5680fd4b5040a/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:28.454388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:28.477314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46561a00-2f47-478a-9343-2960c3569d07 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:28.563724 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46422b1-2f5f-4309-b8d6-f6d06509fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:28.566623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46422b1-2f5f-4309-b8d6-f6d06509fdde admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.567054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46422b1-2f5f-4309-b8d6-f6d06509fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.567472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46422b1-2f5f-4309-b8d6-f6d06509fdde admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.568427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46422b1-2f5f-4309-b8d6-f6d06509fdde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:28.570466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46422b1-2f5f-4309-b8d6-f6d06509fdde admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:28.586328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1181/2351] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:28 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:28.593699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:28.604836 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929e11af-27c7-47a8-9421-95266e7f52e6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:28.610064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:28.610971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.611305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.611588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] PATH_INFO: `/v3/projects/e0f8035f7863430ca4c37beec257ed53/users/127e0f8eee4d4a3eb0d5680fd4b5040a/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.614008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0f8035f7863430ca4c37beec257ed53, user_id=127e0f8eee4d4a3eb0d5680fd4b5040a, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:28.618534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:28.628876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1171/2352] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:28 2022] PUT /identity/v3/projects/e0f8035f7863430ca4c37beec257ed53/users/127e0f8eee4d4a3eb0d5680fd4b5040a/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:28.639717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bce83-9f31-409a-8b0c-6ad0756789be None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.640325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bce83-9f31-409a-8b0c-6ad0756789be None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.641077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9bce83-9f31-409a-8b0c-6ad0756789be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.709591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9d9bce83-9f31-409a-8b0c-6ad0756789be None None] MFA Rules not processed for user `127e0f8eee4d4a3eb0d5680fd4b5040a`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:28.710899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9d9bce83-9f31-409a-8b0c-6ad0756789be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:28.772648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:28.806832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1182/2353] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:28 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:28.817958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:28.819976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:28.920429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e308f7-c6fc-41af-9730-aaf89ea6b779 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:28.926218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104ad116-438e-4397-99ef-410d5310a2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:28.927235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad116-438e-4397-99ef-410d5310a2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:28.927713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad116-438e-4397-99ef-410d5310a2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:28.928240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104ad116-438e-4397-99ef-410d5310a2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:28.944284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ad116-438e-4397-99ef-410d5310a2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:28.950707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104ad116-438e-4397-99ef-410d5310a2a9 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:28.975025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1172/2354] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:28 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:29.771622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bce83-9f31-409a-8b0c-6ad0756789be None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:29.782543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9bce83-9f31-409a-8b0c-6ad0756789be None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:29.787224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:29.788037 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:29.788416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:29.788786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:29.870074 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:29.871539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:29.893242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1183/2355] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:35.182273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076ba0-aed7-4f1b-a3ba-325155967d3c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:35.182905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076ba0-aed7-4f1b-a3ba-325155967d3c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:35.183005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18076ba0-aed7-4f1b-a3ba-325155967d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:35.216703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-18076ba0-aed7-4f1b-a3ba-325155967d3c None None] MFA Rules not processed for user `9ae3a6ce6ab4425ebf1180721c10eb2d`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:35.223114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-18076ba0-aed7-4f1b-a3ba-325155967d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:35.324830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1173/2356] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:36.110902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:36.114076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:36.191442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870c334a-46af-46e5-809c-a580c6576104 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:36.197957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:36.198914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:36.199319 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:36.199531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] PATH_INFO: `/v3/users/bbe6b7c6169d48aea6253c308236e4b3` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:36.200783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbe6b7c6169d48aea6253c308236e4b3)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:36.202880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:36.266180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbe6b7c6169d48aea6253c308236e4b3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:36.275160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbe6b7c6169d48aea6253c308236e4b3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:36.276077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbe6b7c6169d48aea6253c308236e4b3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:36.289751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbe6b7c6169d48aea6253c308236e4b3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:36.296685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:36.299389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbe6b7c6169d48aea6253c308236e4b3'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:36.306100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1184/2357] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:36 2022] DELETE /identity/v3/users/bbe6b7c6169d48aea6253c308236e4b3 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:36.310942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:36.311492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:36.311655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:36.334440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:36.335874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:36.346463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:36.412350 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1174/2358] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:36.418845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:36.420456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:36.498707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c145ef-ce7d-4672-9190-921091a20d2f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:36.513147 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:36.513832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:36.514117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:36.514393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:36.527527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:36.529087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:36.540515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1185/2359] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:36.597556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:36.600087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:36.675057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51593a63-2b45-489b-863e-6e83929cac71 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:36.678977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c04317-aeaf-4e17-94d7-18ffd9b2bf78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:36.679632 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04317-aeaf-4e17-94d7-18ffd9b2bf78 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:36.679945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04317-aeaf-4e17-94d7-18ffd9b2bf78 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:36.680222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c04317-aeaf-4e17-94d7-18ffd9b2bf78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:36.682250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f7c04317-aeaf-4e17-94d7-18ffd9b2bf78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:36.746060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c04317-aeaf-4e17-94d7-18ffd9b2bf78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:36.747560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c04317-aeaf-4e17-94d7-18ffd9b2bf78 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:36.757134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1175/2360] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:36.929908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:36.932007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:36.948716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5c71c0-9ee1-43f3-b7aa-1c5232bdc386 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:36.957567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:36.959601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:36.959791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:36.960117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] PATH_INFO: `/v3/projects/2ef29d7b1c33443087a1d90409dd932c` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:36.962855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef29d7b1c33443087a1d90409dd932c)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:36.963960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.035614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef29d7b1c33443087a1d90409dd932c', 'request_id': 'req-94d7af02-9230-4be6-af30-ae2012f49c54'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:37.060477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef29d7b1c33443087a1d90409dd932c', 'request_id': 'req-94d7af02-9230-4be6-af30-ae2012f49c54'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:37.076710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef29d7b1c33443087a1d90409dd932c', 'request_id': 'req-94d7af02-9230-4be6-af30-ae2012f49c54'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:37.104955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] The token cache is being invalidate because project 2ef29d7b1c33443087a1d90409dd932c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:37.105718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:37.106518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-94d7af02-9230-4be6-af30-ae2012f49c54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:37.108888 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1186/2361] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:36 2022] DELETE /identity/v3/projects/2ef29d7b1c33443087a1d90409dd932c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:37.113947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78c1a1-8c49-4ce9-a638-1ce3b0518c6d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.114422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78c1a1-8c49-4ce9-a638-1ce3b0518c6d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.114799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e78c1a1-8c49-4ce9-a638-1ce3b0518c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.140827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3e78c1a1-8c49-4ce9-a638-1ce3b0518c6d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:37.142700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3e78c1a1-8c49-4ce9-a638-1ce3b0518c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:37.243797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1176/2362] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:37.248984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b6d9-8502-4c25-9008-2cf0b4915ed0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.249460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b6d9-8502-4c25-9008-2cf0b4915ed0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.249553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118b6d9-8502-4c25-9008-2cf0b4915ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.269836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5118b6d9-8502-4c25-9008-2cf0b4915ed0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:37.271225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5118b6d9-8502-4c25-9008-2cf0b4915ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:37.282087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1187/2363] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:37.286829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78c1a1-8c49-4ce9-a638-1ce3b0518c6d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:37.297494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e78c1a1-8c49-4ce9-a638-1ce3b0518c6d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:37.302027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:37.302689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.302974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.303254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.304157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:37.304378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:37.306166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.314419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:37.321975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1177/2364] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:37.326517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b6d9-8502-4c25-9008-2cf0b4915ed0 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:37.337851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118b6d9-8502-4c25-9008-2cf0b4915ed0 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:37.342619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:37.343289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.343583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.343912 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.344670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:37.346583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.376711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1188/2365] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:37 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:37.380764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:37.391051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2b153e-00e9-4628-a02c-81a20e62a156 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:37.448090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:37.448760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.449041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.449414 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.450100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:37.452028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.473518 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1178/2366] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:37 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:37.478234 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:37.488775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39331a07-498e-4dab-bf84-159299195127 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:37.493739 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:37.494416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.494706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.495073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.495779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:37.497391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.510141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1189/2367] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:37.518904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:37.529858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b4c912-325c-40e4-996c-c06e92dd4559 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:37.534695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38c128-a5af-4a30-96da-cc7b7bd02b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:37.535988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38c128-a5af-4a30-96da-cc7b7bd02b4d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.535988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38c128-a5af-4a30-96da-cc7b7bd02b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.535988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38c128-a5af-4a30-96da-cc7b7bd02b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.536772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38c128-a5af-4a30-96da-cc7b7bd02b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:37.538537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38c128-a5af-4a30-96da-cc7b7bd02b4d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.549363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1179/2368] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:37.553897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:37.568676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0027aad-befe-4fdc-91a2-fca4f8b600d1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:37.574343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:37.575248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.575536 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.575853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] PATH_INFO: `/v3/projects/71b2e4cb235e41da9ade19672bdd2e4d/users/ee9bcb3226f240a5951ab9971f6bb654/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.593952 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b2e4cb235e41da9ade19672bdd2e4d, user_id=ee9bcb3226f240a5951ab9971f6bb654, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:37.596787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.614565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1190/2369] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:37 2022] PUT /identity/v3/projects/71b2e4cb235e41da9ade19672bdd2e4d/users/ee9bcb3226f240a5951ab9971f6bb654/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:37.625387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85842aa-888b-41b1-9a1e-8a5a79e91194 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.625680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85842aa-888b-41b1-9a1e-8a5a79e91194 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.625960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85842aa-888b-41b1-9a1e-8a5a79e91194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.655643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c85842aa-888b-41b1-9a1e-8a5a79e91194 None None] MFA Rules not processed for user `ee9bcb3226f240a5951ab9971f6bb654`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:37.656913 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c85842aa-888b-41b1-9a1e-8a5a79e91194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:37.723113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1180/2370] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:37.733831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:37.735877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:37.756684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:37.798947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b799c2-f3ef-4832-aed3-31b884fd0522 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:37.803209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:37.803936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:37.804291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:37.804638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:37.813433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:37.814947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:37.824734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1191/2371] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:37 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:40.377899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf3ff-d6e3-44d3-a1ba-40f8c4aee44c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:40.378369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf3ff-d6e3-44d3-a1ba-40f8c4aee44c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:40.378445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccbf3ff-d6e3-44d3-a1ba-40f8c4aee44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:40.398083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-eccbf3ff-d6e3-44d3-a1ba-40f8c4aee44c None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:40.399332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eccbf3ff-d6e3-44d3-a1ba-40f8c4aee44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:40.462605 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1181/2372] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:27:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:40.475188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:40.476850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:40.563421 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55d7a17-32a8-4201-b5a6-8a37227e8d04 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:40.568804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:40.569471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:40.569759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:40.570301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:40.578756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:40.580396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:40.588883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1192/2373] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:27:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:27:41.280498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe70715-2b03-4b85-89ef-18590f49701b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:41.280857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe70715-2b03-4b85-89ef-18590f49701b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:41.281064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe70715-2b03-4b85-89ef-18590f49701b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:41.302296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bfe70715-2b03-4b85-89ef-18590f49701b None None] MFA Rules not processed for user `ee9bcb3226f240a5951ab9971f6bb654`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:41.303528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bfe70715-2b03-4b85-89ef-18590f49701b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:41.311275 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1182/2374] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:42.235528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:42.237051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:42.329963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcfb7bf-e910-48b2-b3e8-476bc6e9fb39 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:42.337734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:42.339208 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:42.339709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:42.340192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] PATH_INFO: `/v3/users/2f5f96917087414fa11317541ef7ef13` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:42.341614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f5f96917087414fa11317541ef7ef13)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:42.344119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:42.418381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f5f96917087414fa11317541ef7ef13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.423562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:42.434656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f5f96917087414fa11317541ef7ef13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.435944 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f5f96917087414fa11317541ef7ef13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.471108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f5f96917087414fa11317541ef7ef13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.485750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f5f96917087414fa11317541ef7ef13'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.493360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1193/2375] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:42 2022] DELETE /identity/v3/users/2f5f96917087414fa11317541ef7ef13 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:42.497911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:42.498460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:42.498938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:42.522607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:42.535709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:42.537444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:42.594484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:42.596842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:42.659253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1183/2376] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:42.673188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:42.678882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:42.702517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51032166-7f42-424e-a9bc-93b53ab68efe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:42.712406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:42.713316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:42.713584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:42.714061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] PATH_INFO: `/v3/users/1d26db3bcf154dd98e5e62dc47a41dd9` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:42.716608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d26db3bcf154dd98e5e62dc47a41dd9)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:42.718758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:42.791935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74507fc6-4212-4be0-aaf2-da12a2df4859 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:42.796508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f97ba-ed84-4996-aa2b-5248be462d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:42.797310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f97ba-ed84-4996-aa2b-5248be462d1a service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:42.797602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f97ba-ed84-4996-aa2b-5248be462d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:42.797995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f97ba-ed84-4996-aa2b-5248be462d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:42.813094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d26db3bcf154dd98e5e62dc47a41dd9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.813340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f97ba-ed84-4996-aa2b-5248be462d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:42.814090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f97ba-ed84-4996-aa2b-5248be462d1a service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:42.822793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d26db3bcf154dd98e5e62dc47a41dd9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.824869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d26db3bcf154dd98e5e62dc47a41dd9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.830967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1184/2377] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:42.860101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d26db3bcf154dd98e5e62dc47a41dd9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.877478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d26db3bcf154dd98e5e62dc47a41dd9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:42.902908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1194/2378] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:42 2022] DELETE /identity/v3/users/1d26db3bcf154dd98e5e62dc47a41dd9 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:42.904667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:42.905041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:42.905334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:42.951587 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:42.953556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:43.079227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1185/2379] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:43.089680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:43.091426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:43.169564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:43.175252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24684eb-a632-40b3-a949-08658997e156 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:43.181326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:43.196180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:43.196626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:43.196984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:43.212221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:43.213711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:43.227915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1195/2380] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:43.282269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:43.283710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:43.297911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22bb5c1-3ae1-431d-8cf4-fb913b79a7ec None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:43.303565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:43.304830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:43.305107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:43.305901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] PATH_INFO: `/v3/projects/b6431cf402df4f76ba422a5b8f9fea4b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:43.307185 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6431cf402df4f76ba422a5b8f9fea4b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:43.309652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:43.378865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:43.384647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6431cf402df4f76ba422a5b8f9fea4b', 'request_id': 'req-3dff020a-84e2-48d4-95e0-223c8298ea40'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.408979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6431cf402df4f76ba422a5b8f9fea4b', 'request_id': 'req-3dff020a-84e2-48d4-95e0-223c8298ea40'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.432584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6431cf402df4f76ba422a5b8f9fea4b', 'request_id': 'req-3dff020a-84e2-48d4-95e0-223c8298ea40'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.459377 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] The token cache is being invalidate because project b6431cf402df4f76ba422a5b8f9fea4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:43.460010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.462012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.465062 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1186/2381] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:43 2022] DELETE /identity/v3/projects/b6431cf402df4f76ba422a5b8f9fea4b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:43.469621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:43.485064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beaf254-0309-4224-8ac2-29e9012903b0 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:43.566139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:43.567031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:43.567313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:43.568148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] PATH_INFO: `/v3/users/5e3c9034c4ab4d548d8f0ea26094b28b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:43.569951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3c9034c4ab4d548d8f0ea26094b28b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:43.572286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:43.628743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:43.630863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:43.660331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dff020a-84e2-48d4-95e0-223c8298ea40 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:43.677093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:43.679733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:43.681210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:43.681610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] PATH_INFO: `/v3/projects/cf8af4dee7e84c5f9f61deaea32ff163` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:43.683979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf8af4dee7e84c5f9f61deaea32ff163)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:43.687812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:43.697103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3c9034c4ab4d548d8f0ea26094b28b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.714660 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3c9034c4ab4d548d8f0ea26094b28b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.715772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3c9034c4ab4d548d8f0ea26094b28b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.740217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3c9034c4ab4d548d8f0ea26094b28b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.752587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3c9034c4ab4d548d8f0ea26094b28b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.768345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf8af4dee7e84c5f9f61deaea32ff163', 'request_id': 'req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.775128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf8af4dee7e84c5f9f61deaea32ff163', 'request_id': 'req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.779225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1196/2382] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:43 2022] DELETE /identity/v3/users/5e3c9034c4ab4d548d8f0ea26094b28b => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:43.793055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf8af4dee7e84c5f9f61deaea32ff163', 'request_id': 'req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.818709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] The token cache is being invalidate because project cf8af4dee7e84c5f9f61deaea32ff163 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:43.818938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.830698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:43.843145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1187/2383] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:43 2022] DELETE /identity/v3/projects/cf8af4dee7e84c5f9f61deaea32ff163 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:44.305226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:44.307353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:44.388606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae54a5d-392b-4cae-9ded-8845328da998 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:44.393370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:44.394338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:44.394704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:44.395050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] PATH_INFO: `/v3/projects/4f24942bdfd04c77a8df36a19bd3e820` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:44.396379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f24942bdfd04c77a8df36a19bd3e820)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:44.403954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:44.418977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:44.458254 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f24942bdfd04c77a8df36a19bd3e820', 'request_id': 'req-98978678-34a3-4cf5-9818-f7b5b386c370'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.466621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f24942bdfd04c77a8df36a19bd3e820', 'request_id': 'req-98978678-34a3-4cf5-9818-f7b5b386c370'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.473487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f24942bdfd04c77a8df36a19bd3e820', 'request_id': 'req-98978678-34a3-4cf5-9818-f7b5b386c370'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.484965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] The token cache is being invalidate because project 4f24942bdfd04c77a8df36a19bd3e820 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:44.485272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.485929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.487499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1197/2384] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:44 2022] DELETE /identity/v3/projects/4f24942bdfd04c77a8df36a19bd3e820 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:44.491418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:44.493393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:44.560423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e86b8c-2cbd-4955-8b99-4f0b5ff65eaa admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:44.637404 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:44.638298 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:44.638606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:44.638882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] PATH_INFO: `/v3/users/127e0f8eee4d4a3eb0d5680fd4b5040a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:44.640214 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] RBAC: Authorizing `identity:delete_user(user_id=127e0f8eee4d4a3eb0d5680fd4b5040a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:44.642233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:44.645195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:44.698948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '127e0f8eee4d4a3eb0d5680fd4b5040a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.707525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '127e0f8eee4d4a3eb0d5680fd4b5040a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.708198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '127e0f8eee4d4a3eb0d5680fd4b5040a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.720307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '127e0f8eee4d4a3eb0d5680fd4b5040a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.741352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e4dd595c-11ba-47d9-88ce-6a5f17306967 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '127e0f8eee4d4a3eb0d5680fd4b5040a'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:44.747508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1188/2385] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:44 2022] DELETE /identity/v3/users/127e0f8eee4d4a3eb0d5680fd4b5040a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:45.100160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:45.103856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:45.136178 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98978678-34a3-4cf5-9818-f7b5b386c370 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:45.228585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:45.229465 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.229749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.230030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] PATH_INFO: `/v3/projects/e0f8035f7863430ca4c37beec257ed53` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.231326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0f8035f7863430ca4c37beec257ed53)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:45.233222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:45.316310 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0f8035f7863430ca4c37beec257ed53', 'request_id': 'req-e93f8b91-4aca-4663-82ee-9ce8ed82c428'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:45.334247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0f8035f7863430ca4c37beec257ed53', 'request_id': 'req-e93f8b91-4aca-4663-82ee-9ce8ed82c428'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:45.344306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0f8035f7863430ca4c37beec257ed53', 'request_id': 'req-e93f8b91-4aca-4663-82ee-9ce8ed82c428'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:45.359206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] The token cache is being invalidate because project e0f8035f7863430ca4c37beec257ed53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:45.359638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:45.360562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e93f8b91-4aca-4663-82ee-9ce8ed82c428 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:45.362776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1198/2386] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:45 2022] DELETE /identity/v3/projects/e0f8035f7863430ca4c37beec257ed53 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:45.369750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877e93e-6afa-4225-8024-8b9952d44036 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.370291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877e93e-6afa-4225-8024-8b9952d44036 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.370649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a877e93e-6afa-4225-8024-8b9952d44036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.397986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a877e93e-6afa-4225-8024-8b9952d44036 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:45.399418 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a877e93e-6afa-4225-8024-8b9952d44036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:45.477931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1189/2387] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:45.487015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbe88-8acc-4e68-8d6b-c8ed382780dd None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.487928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbe88-8acc-4e68-8d6b-c8ed382780dd None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.488384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fbe88-8acc-4e68-8d6b-c8ed382780dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.520308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-835fbe88-8acc-4e68-8d6b-c8ed382780dd None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:45.528701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-835fbe88-8acc-4e68-8d6b-c8ed382780dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:45.545072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1199/2388] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:45.549770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877e93e-6afa-4225-8024-8b9952d44036 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:45.572131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a877e93e-6afa-4225-8024-8b9952d44036 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:45.577014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:45.577684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.577966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.578241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.578967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:45.579457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:45.581106 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:45.599119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1190/2389] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:45.600215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbe88-8acc-4e68-8d6b-c8ed382780dd None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:45.609972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fbe88-8acc-4e68-8d6b-c8ed382780dd None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:45.614782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:45.615586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.615911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.616194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.616900 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:45.619426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:45.657976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1200/2390] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:45 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:27:45.662044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:45.674162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5b08fd-60f8-4f32-b72e-c0fd58267bc5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:45.734645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:45.735208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.735479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.735788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.736489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:45.738431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:45.753198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:45.762564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1191/2391] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:45 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:45.766792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:45.772364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:45.778940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af48e7a-1027-4667-8d5d-7df26c3f1320 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:45.786981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:45.788129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.788610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.789047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.790165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:45.792070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:45.803270 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1201/2392] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:45.807560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:45.818050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665e6b-0f41-4b47-83b2-d94dbb2f4ea9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:45.823583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdca518c-d2a4-49b8-ab67-0e114d0c8f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:45.824278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca518c-d2a4-49b8-ab67-0e114d0c8f2d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.824562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca518c-d2a4-49b8-ab67-0e114d0c8f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.824841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdca518c-d2a4-49b8-ab67-0e114d0c8f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.825581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca518c-d2a4-49b8-ab67-0e114d0c8f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:45.827234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdca518c-d2a4-49b8-ab67-0e114d0c8f2d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:45.838346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1192/2393] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:45.842648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:45.852701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d39ed91-f1e5-42ea-aa72-ff81ea6231f9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:45.857448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:45.858350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.858640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.858924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] PATH_INFO: `/v3/projects/181dcddf62c141afb92beb902fa9b311/users/f15d84abf9e3448baff526711cad9471/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.875142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=181dcddf62c141afb92beb902fa9b311, user_id=f15d84abf9e3448baff526711cad9471, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:45.877892 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:45.896297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1202/2394] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:45 2022] PUT /identity/v3/projects/181dcddf62c141afb92beb902fa9b311/users/f15d84abf9e3448baff526711cad9471/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:45.916700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e98a68-ef61-4bfa-bf24-055b8c213fd0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:45.917183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e98a68-ef61-4bfa-bf24-055b8c213fd0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:45.917675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e98a68-ef61-4bfa-bf24-055b8c213fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:45.960570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c6e98a68-ef61-4bfa-bf24-055b8c213fd0 None None] MFA Rules not processed for user `f15d84abf9e3448baff526711cad9471`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:45.962490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c6e98a68-ef61-4bfa-bf24-055b8c213fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:46.037283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1193/2395] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:46.041404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.131099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6066233-e814-4e30-8ff9-6aec3c0d6fa6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.136641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.138002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.138396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.138781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.139721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:46.141789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:46.168999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1203/2396] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:46 2022] POST /identity/v3/projects => generated 387 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:46.173009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e98a68-ef61-4bfa-bf24-055b8c213fd0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.187076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e98a68-ef61-4bfa-bf24-055b8c213fd0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.205201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.207022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:46.310735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.311443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.311608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.311868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.312857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:46.314839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:46.363941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007027e0-e5ff-4e3d-ad33-0c38f710167c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.377317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1194/2397] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:46 2022] POST /identity/v3/users => generated 300 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:46.378844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.381278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.382346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.382637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.382917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] PATH_INFO: `/v3/users/83047d45a7f74e468e4635ecb391cf7d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.391389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] RBAC: Authorizing `identity:delete_user(user_id=83047d45a7f74e468e4635ecb391cf7d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:46.393135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:46.399138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b9349e-f347-42b0-9f09-2c2864976ef9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.412576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.412576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.412902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.413165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.413897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:46.415564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:46.441265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1195/2398] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:46.446354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.467574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b24e590-2f7e-4b28-93f7-857a8347682b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.475719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.478108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.481063 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.481446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] PATH_INFO: `/v3/projects/2a26cf8a05bc42d28d953364dddb8d24/users/351775f9d453420dba8370385ae97516/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.505593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83047d45a7f74e468e4635ecb391cf7d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:46.510304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a26cf8a05bc42d28d953364dddb8d24, user_id=351775f9d453420dba8370385ae97516, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:46.513542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:46.520153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83047d45a7f74e468e4635ecb391cf7d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:46.520403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83047d45a7f74e468e4635ecb391cf7d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:46.521988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:46.551829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83047d45a7f74e468e4635ecb391cf7d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:46.558375 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1196/2399] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:46 2022] PUT /identity/v3/projects/2a26cf8a05bc42d28d953364dddb8d24/users/351775f9d453420dba8370385ae97516/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:46.562410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.564371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:46.575303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-8fe45e4b-0043-4751-af74-6a854feff088 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83047d45a7f74e468e4635ecb391cf7d'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:46.586611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1204/2400] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:46 2022] DELETE /identity/v3/users/83047d45a7f74e468e4635ecb391cf7d => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:46.591103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.592797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.592797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.619446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:46.622049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:46.622755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:46.684716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceadbe-1a4d-48c8-97e3-ef343685ae23 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.728028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1205/2401] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:46.735164 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.736787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:46.746105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.746785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.747253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.747538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.748399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:46.750027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:46.761843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1197/2402] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:46 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:46.765805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:46.786002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1782c564-f8ae-4031-a8d6-e74e6735aef1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.801134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7045be-e908-4904-bcb2-9af29e6a46f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.801933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7045be-e908-4904-bcb2-9af29e6a46f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.802307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7045be-e908-4904-bcb2-9af29e6a46f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.802723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7045be-e908-4904-bcb2-9af29e6a46f5 admin admin] PATH_INFO: `/v3/projects/2a26cf8a05bc42d28d953364dddb8d24/users/351775f9d453420dba8370385ae97516/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.810606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7045be-e908-4904-bcb2-9af29e6a46f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a26cf8a05bc42d28d953364dddb8d24, user_id=351775f9d453420dba8370385ae97516, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:46.813630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7045be-e908-4904-bcb2-9af29e6a46f5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:46.847128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1198/2403] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:46 2022] PUT /identity/v3/projects/2a26cf8a05bc42d28d953364dddb8d24/users/351775f9d453420dba8370385ae97516/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:46.850143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d54c16e-513e-4822-b8ff-8d33fecad2ac None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:46.859691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91224046-db15-4307-915a-f12c7795a8df None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.860008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91224046-db15-4307-915a-f12c7795a8df None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.860289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91224046-db15-4307-915a-f12c7795a8df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:46.925621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-91224046-db15-4307-915a-f12c7795a8df None None] MFA Rules not processed for user `351775f9d453420dba8370385ae97516`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:46.925621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-91224046-db15-4307-915a-f12c7795a8df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:46.933003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:46.933681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:46.933968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:46.934247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:47.004941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:47.033506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:47.035292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:47.060435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1206/2404] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:47.075811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1199/2405] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:46 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:47.081991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:47.083538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:47.170919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fa08da-b349-4102-a044-793af0db0379 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:47.174990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc88d0d-73e3-428f-8863-df96ba79e90b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:47.175801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc88d0d-73e3-428f-8863-df96ba79e90b service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:47.176372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc88d0d-73e3-428f-8863-df96ba79e90b service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:47.176907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc88d0d-73e3-428f-8863-df96ba79e90b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:47.189440 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc88d0d-73e3-428f-8863-df96ba79e90b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:47.191391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc88d0d-73e3-428f-8863-df96ba79e90b service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:47.205522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1207/2406] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:47.382693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91224046-db15-4307-915a-f12c7795a8df None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:47.396010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91224046-db15-4307-915a-f12c7795a8df None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:47.406468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:47.406468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:47.406468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:47.406468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] PATH_INFO: `/v3/users/f15d84abf9e3448baff526711cad9471` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:47.406894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15d84abf9e3448baff526711cad9471)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:47.408714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:47.483149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:47.489980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15d84abf9e3448baff526711cad9471'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.502761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15d84abf9e3448baff526711cad9471'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.513051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15d84abf9e3448baff526711cad9471'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.531955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15d84abf9e3448baff526711cad9471'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.556954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15d84abf9e3448baff526711cad9471'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.567956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1200/2407] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:47 2022] DELETE /identity/v3/users/f15d84abf9e3448baff526711cad9471 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:47.572599 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:47.573103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:47.573223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:47.620928 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:47.627008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:47.654501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:47.656028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:47.717264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1208/2408] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:47.753661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:47.755199 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:47.796046 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a861cf-3678-4ba9-80ed-102c73d36818 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:47.805950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:47.806904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:47.807942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:47.808609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] PATH_INFO: `/v3/projects/17f33598fdb14edaa577b108c1ea87fa` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:47.811491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=17f33598fdb14edaa577b108c1ea87fa)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:47.816529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:47.901589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17f33598fdb14edaa577b108c1ea87fa', 'request_id': 'req-2c1262c5-811a-4ff7-947f-19b8049c54bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.902502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4396f2-32d6-4709-b0d6-a5a66ec4d55d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:47.908817 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17f33598fdb14edaa577b108c1ea87fa', 'request_id': 'req-2c1262c5-811a-4ff7-947f-19b8049c54bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.918339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17f33598fdb14edaa577b108c1ea87fa', 'request_id': 'req-2c1262c5-811a-4ff7-947f-19b8049c54bb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.932472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] The token cache is being invalidate because project 17f33598fdb14edaa577b108c1ea87fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:47.933076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.933930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-2c1262c5-811a-4ff7-947f-19b8049c54bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:47.936088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1201/2409] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:47 2022] DELETE /identity/v3/projects/17f33598fdb14edaa577b108c1ea87fa => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:47.941217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946014ff-6c03-4367-bb7d-de37f25183a2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:47.941812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946014ff-6c03-4367-bb7d-de37f25183a2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:47.942308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946014ff-6c03-4367-bb7d-de37f25183a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:47.958211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:47.977609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-946014ff-6c03-4367-bb7d-de37f25183a2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:47.978875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-946014ff-6c03-4367-bb7d-de37f25183a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:47.993441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:47.994757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:47.995240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:47.995717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:47.997761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:48.052545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:48.090332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1202/2410] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:48.092398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77e697-b3f3-4edc-9d6b-a403fbabe9cc None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.092926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77e697-b3f3-4edc-9d6b-a403fbabe9cc None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.093304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f77e697-b3f3-4edc-9d6b-a403fbabe9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.106420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.108330 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.113532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1f77e697-b3f3-4edc-9d6b-a403fbabe9cc None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:48.114826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1f77e697-b3f3-4edc-9d6b-a403fbabe9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:48.120142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1209/2411] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:48.135226 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1203/2412] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:48.136634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.154119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904cc6a0-355b-43cc-b0ba-6f622790504d service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.160623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.161745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.162315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.162754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.164064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:48.164672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.167826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.186094 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1210/2413] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:48.200335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77e697-b3f3-4edc-9d6b-a403fbabe9cc None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.227017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f77e697-b3f3-4edc-9d6b-a403fbabe9cc None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.235873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.236606 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.237391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.237391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.240357 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.244869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.282401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1204/2414] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:48 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:48.286685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.299550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b478c4f-8282-407c-996f-f8762badce27 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.370727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.371943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.372451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.372934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.373872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.376052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.403522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1211/2415] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:48 2022] POST /identity/v3/users => generated 291 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:48.407219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.417942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938240f4-d4f9-47f4-b102-77e9e8fbddd9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.430737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.431514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.431923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.432767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.433630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.444558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.475499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1205/2416] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:48.480611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.496252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0033e3-f651-4dca-89f4-360e35709694 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.536549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d990cee-c3be-41a8-9e61-0caae7ed8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.537406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d990cee-c3be-41a8-9e61-0caae7ed8720 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.537798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d990cee-c3be-41a8-9e61-0caae7ed8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.538186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d990cee-c3be-41a8-9e61-0caae7ed8720 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.539042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d990cee-c3be-41a8-9e61-0caae7ed8720 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.540884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d990cee-c3be-41a8-9e61-0caae7ed8720 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.556142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1212/2417] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:48 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:48.558671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.571698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910dd2af-26f9-487a-b20f-20a3fe4c154e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.576092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.576984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.577260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.577593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] PATH_INFO: `/v3/projects/1bd64979a2bc4ca6be57262f4387920e/users/9476865e0b114996b2c0ffce9fca16a2/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.595021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bd64979a2bc4ca6be57262f4387920e, user_id=9476865e0b114996b2c0ffce9fca16a2, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.598255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.623067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1206/2418] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:48 2022] PUT /identity/v3/projects/1bd64979a2bc4ca6be57262f4387920e/users/9476865e0b114996b2c0ffce9fca16a2/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:48.624736 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.625244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.625343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.649633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:48.652969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:48.695022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.697778 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:48.727716 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:48.752628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1213/2419] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:48.762146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:48.763559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:48.848793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab7045d-fa45-45e9-b088-2c7f1741b62e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.854346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.855015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.856735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.856735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.861017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723ae56-313f-403e-a6b0-9bc7d30ed82d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:48.869078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.871083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.873498 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:48.874406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:48.874848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:48.874962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] PATH_INFO: `/v3/projects/181dcddf62c141afb92beb902fa9b311` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:48.876225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=181dcddf62c141afb92beb902fa9b311)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:48.878164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:48.886854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1214/2420] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:27:49.009821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '181dcddf62c141afb92beb902fa9b311', 'request_id': 'req-f6c54c59-59cd-4359-92d2-7508146fccfc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.016773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '181dcddf62c141afb92beb902fa9b311', 'request_id': 'req-f6c54c59-59cd-4359-92d2-7508146fccfc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.029393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '181dcddf62c141afb92beb902fa9b311', 'request_id': 'req-f6c54c59-59cd-4359-92d2-7508146fccfc'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.054312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] The token cache is being invalidate because project 181dcddf62c141afb92beb902fa9b311 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:49.054609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.057619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.061183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1207/2421] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:48 2022] DELETE /identity/v3/projects/181dcddf62c141afb92beb902fa9b311 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:49.065485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:49.079285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c290ec-32fd-478e-8d12-ad2eedde2662 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:49.221714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:49.222619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:49.222917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:49.223191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] PATH_INFO: `/v3/users/351775f9d453420dba8370385ae97516` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:49.224820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] RBAC: Authorizing `identity:delete_user(user_id=351775f9d453420dba8370385ae97516)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:49.227511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:49.294949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '351775f9d453420dba8370385ae97516'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.303534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '351775f9d453420dba8370385ae97516'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.304324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '351775f9d453420dba8370385ae97516'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.328477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '351775f9d453420dba8370385ae97516'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.337985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-05bdae21-c130-4660-8cc3-3026e5c6397d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '351775f9d453420dba8370385ae97516'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:49.349709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1215/2422] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:49 2022] DELETE /identity/v3/users/351775f9d453420dba8370385ae97516 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:49.753487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:49.755826 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:49.782244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c54c59-59cd-4359-92d2-7508146fccfc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:49.821097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:49.939475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:49.940583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:49.942955 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:49.943351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] PATH_INFO: `/v3/projects/2a26cf8a05bc42d28d953364dddb8d24` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:49.949197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a26cf8a05bc42d28d953364dddb8d24)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:49.953865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:49.985269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:49.986120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:49.986807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.049665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] MFA Rules not processed for user `9476865e0b114996b2c0ffce9fca16a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:50.051369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:50.065136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a26cf8a05bc42d28d953364dddb8d24', 'request_id': 'req-9adc2ab2-821b-4563-a201-a98e37905009'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:50.071884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:50.079627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a26cf8a05bc42d28d953364dddb8d24', 'request_id': 'req-9adc2ab2-821b-4563-a201-a98e37905009'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:50.088994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a26cf8a05bc42d28d953364dddb8d24', 'request_id': 'req-9adc2ab2-821b-4563-a201-a98e37905009'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:50.107064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] The token cache is being invalidate because project 2a26cf8a05bc42d28d953364dddb8d24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:50.107362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:50.112426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-9adc2ab2-821b-4563-a201-a98e37905009 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:50.122485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1208/2423] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:49 2022] DELETE /identity/v3/projects/2a26cf8a05bc42d28d953364dddb8d24 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:50.125705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc6e13-5300-42e2-be49-272359f19878 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.126232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc6e13-5300-42e2-be49-272359f19878 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.126746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc6e13-5300-42e2-be49-272359f19878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.195479 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-92dc6e13-5300-42e2-be49-272359f19878 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:50.198728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-92dc6e13-5300-42e2-be49-272359f19878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:50.216829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1216/2424] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:50.231151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:50.232745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:50.316110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1209/2425] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:50.325769 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a78d34-1502-48d5-b412-668d19f10bd6 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.326114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a78d34-1502-48d5-b412-668d19f10bd6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.327103 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a78d34-1502-48d5-b412-668d19f10bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.365914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b7a78d34-1502-48d5-b412-668d19f10bd6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:50.367186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b7a78d34-1502-48d5-b412-668d19f10bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:50.367585 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b79c56a-ec54-4c20-ab90-97cb2d708484 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:50.389971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:50.391101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.391404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.391776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.394444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1210/2426] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:50.399860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a78d34-1502-48d5-b412-668d19f10bd6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:50.403949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:50.406171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:50.418216 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a78d34-1502-48d5-b412-668d19f10bd6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:50.421272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1217/2427] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:27:50.438748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:50.439431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.439713 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.440009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.440751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:50.441049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:50.442683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:50.491081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1211/2428] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:50.496071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:50.527666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abbfda-1a68-4219-ac32-44938fd9197a service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:50.533905 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:50.535126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.535420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.536153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.537600 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:50.540268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:50.572381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1218/2429] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:50 2022] POST /identity/v3/projects => generated 371 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:50.582910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:50.593914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a86cfc-1af9-416e-a19f-9e67aeabc843 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:50.624839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:50.669236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:50.669924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.670213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.670745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.671445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:50.673424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:50.696881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1212/2430] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:50 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:50.701868 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:50.714082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e85ff12-387f-426a-b571-07a72a637346 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:50.718965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:50.719836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.720087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.720369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.721231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:50.722863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:50.739348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1219/2431] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:50.741034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:50.753034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05a32ab-e42b-4ae9-bd68-797529bfd5b6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:50.757900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e297e958-b30a-43bd-a06c-1e7edb1585db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:50.758742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e958-b30a-43bd-a06c-1e7edb1585db admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.759013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e958-b30a-43bd-a06c-1e7edb1585db admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.759294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e297e958-b30a-43bd-a06c-1e7edb1585db admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.760075 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e958-b30a-43bd-a06c-1e7edb1585db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:50.761731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e297e958-b30a-43bd-a06c-1e7edb1585db admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:50.774921 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1213/2432] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:50.780714 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:50.795046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e855634-2e9d-4f47-868c-1efd49ceb93e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:50.800733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:50.801669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.802209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.802481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] PATH_INFO: `/v3/projects/22e8ba5426a148e9ad729764df778abb/users/2da5544a66444873b39a80017dda34e5/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.817869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e8ba5426a148e9ad729764df778abb, user_id=2da5544a66444873b39a80017dda34e5, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:50.821187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:50.835167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1220/2433] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:50 2022] PUT /identity/v3/projects/22e8ba5426a148e9ad729764df778abb/users/2da5544a66444873b39a80017dda34e5/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:50.845948 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b27c4-8fe1-471e-9d55-d9f61cc46ce5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:50.846262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b27c4-8fe1-471e-9d55-d9f61cc46ce5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:50.846548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18b27c4-8fe1-471e-9d55-d9f61cc46ce5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:50.874662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b18b27c4-8fe1-471e-9d55-d9f61cc46ce5 None None] MFA Rules not processed for user `2da5544a66444873b39a80017dda34e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:50.875910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b18b27c4-8fe1-471e-9d55-d9f61cc46ce5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:50.943160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1214/2434] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:50 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:50.954800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:51.020564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d0a7fd-2ed9-4c35-bdcd-1dcb72a59c6c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:51.024777 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f50f9-5a64-4dc3-90b2-930f4ca7cd41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:51.025557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f50f9-5a64-4dc3-90b2-930f4ca7cd41 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:51.025848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f50f9-5a64-4dc3-90b2-930f4ca7cd41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:51.026128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f50f9-5a64-4dc3-90b2-930f4ca7cd41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:51.031363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:51.036030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f50f9-5a64-4dc3-90b2-930f4ca7cd41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:51.037904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f50f9-5a64-4dc3-90b2-930f4ca7cd41 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:51.047779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1221/2435] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:50 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:51.066789 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b27c4-8fe1-471e-9d55-d9f61cc46ce5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:51.135253 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18b27c4-8fe1-471e-9d55-d9f61cc46ce5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:51.139844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:51.140715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:51.141015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:51.141265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] PATH_INFO: `/v3/users/2da5544a66444873b39a80017dda34e5` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:51.142459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2da5544a66444873b39a80017dda34e5)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:51.144407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:51.184692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:51.202654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2da5544a66444873b39a80017dda34e5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:51.213724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2da5544a66444873b39a80017dda34e5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:51.214565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2da5544a66444873b39a80017dda34e5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:51.223314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2da5544a66444873b39a80017dda34e5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:51.235776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2da5544a66444873b39a80017dda34e5'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:51.241913 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1215/2436] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:27:51 2022] DELETE /identity/v3/users/2da5544a66444873b39a80017dda34e5 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:51.246537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f6bd8-90a7-4e47-bcc6-c7216335468a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:51.246951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f6bd8-90a7-4e47-bcc6-c7216335468a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:51.247246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f6bd8-90a7-4e47-bcc6-c7216335468a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:51.265543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-572f6bd8-90a7-4e47-bcc6-c7216335468a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:51.266773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-572f6bd8-90a7-4e47-bcc6-c7216335468a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:51.346632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1222/2437] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:51.362330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:51.363928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:51.430990 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85d0371-0893-479c-a453-342b61816252 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:51.436580 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:51.438310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:51.438822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:51.439356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:51.454416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:51.456392 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:51.468989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1216/2438] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:51.856632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39a74-2f5b-4839-9143-8bd423df76b4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:51.856933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39a74-2f5b-4839-9143-8bd423df76b4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:51.857218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e39a74-2f5b-4839-9143-8bd423df76b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:51.876068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-88e39a74-2f5b-4839-9143-8bd423df76b4 None None] MFA Rules not processed for user `9476865e0b114996b2c0ffce9fca16a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:51.877285 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-88e39a74-2f5b-4839-9143-8bd423df76b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:51.936891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1223/2439] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:51 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:52.694912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:52.696700 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:52.731985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf634290-3a9c-4493-a8e7-2a64370cffaf service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:52.746940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:52.748385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:52.748670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:52.748946 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] PATH_INFO: `/v3/projects/22e8ba5426a148e9ad729764df778abb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:52.751348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e8ba5426a148e9ad729764df778abb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:52.753305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:52.855170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e8ba5426a148e9ad729764df778abb', 'request_id': 'req-f3686b27-3fe8-4411-b013-3039a4e9e50c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:52.870136 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e8ba5426a148e9ad729764df778abb', 'request_id': 'req-f3686b27-3fe8-4411-b013-3039a4e9e50c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:52.882820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e8ba5426a148e9ad729764df778abb', 'request_id': 'req-f3686b27-3fe8-4411-b013-3039a4e9e50c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:52.897028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] The token cache is being invalidate because project 22e8ba5426a148e9ad729764df778abb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:27:52.897595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:52.898549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f3686b27-3fe8-4411-b013-3039a4e9e50c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:27:52.901506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1217/2440] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:27:52 2022] DELETE /identity/v3/projects/22e8ba5426a148e9ad729764df778abb => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:52.911941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bc6fe-4d8b-4aab-a1f4-9ec481397075 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:52.912394 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bc6fe-4d8b-4aab-a1f4-9ec481397075 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:52.912475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bc6fe-4d8b-4aab-a1f4-9ec481397075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:52.967148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b87bc6fe-4d8b-4aab-a1f4-9ec481397075 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:52.968436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b87bc6fe-4d8b-4aab-a1f4-9ec481397075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:53.126718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1224/2441] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:53.134556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85bf5-a644-48d6-b774-58afac7b2b77 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.135081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85bf5-a644-48d6-b774-58afac7b2b77 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.135509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c85bf5-a644-48d6-b774-58afac7b2b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.161864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f0c85bf5-a644-48d6-b774-58afac7b2b77 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:53.163152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f0c85bf5-a644-48d6-b774-58afac7b2b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:53.181777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1218/2442] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:53.186135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bc6fe-4d8b-4aab-a1f4-9ec481397075 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.203548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bc6fe-4d8b-4aab-a1f4-9ec481397075 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:53.209142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:53.209830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.210119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.210399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.211669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:27:53.212027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:53.213663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:53.234996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1225/2443] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:27:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:27:53.239714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c85bf5-a644-48d6-b774-58afac7b2b77 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.253746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c85bf5-a644-48d6-b774-58afac7b2b77 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:53.261564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:53.262740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.263196 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.263538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.265007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:53.267682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:53.306683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1219/2444] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:53 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:53.308836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.324025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e2b25-fbd7-4d6d-94ae-4a8bea4be8e0 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:53.353959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:53.400766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:53.401450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.401749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.402035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.402789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:53.404754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:53.433582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1226/2445] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:53 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:53.437487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.453579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabffe2-86e3-49f6-9ea3-ecf4c5061be4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:53.462303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:53.463001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.463319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.463600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.465313 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:53.467022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:53.481419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:53.485169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1220/2446] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:53.486500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.515963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d309af1-5687-427c-83a1-116b4b3ded2b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:53.520106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761202a-8c74-48f0-9a67-1fc64ce6ecd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:53.521172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761202a-8c74-48f0-9a67-1fc64ce6ecd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.521172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761202a-8c74-48f0-9a67-1fc64ce6ecd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.521434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761202a-8c74-48f0-9a67-1fc64ce6ecd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.522359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761202a-8c74-48f0-9a67-1fc64ce6ecd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:53.524849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761202a-8c74-48f0-9a67-1fc64ce6ecd8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:53.550809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.551008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1227/2447] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:53 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:53.569215 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e63ef3-018b-4564-9ac0-c701473cd029 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:53.578984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:53.580054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.580406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.581506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] PATH_INFO: `/v3/projects/90de1322f74d4967b44b0edfbc864ebd/users/2fc6e80da35949f394a257abf01fa8ca/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.610718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90de1322f74d4967b44b0edfbc864ebd, user_id=2fc6e80da35949f394a257abf01fa8ca, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:53.613933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:53.647861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2bf42-f6fb-495e-b12c-de99f432ca83 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.648438 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2bf42-f6fb-495e-b12c-de99f432ca83 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.649005 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2bf42-f6fb-495e-b12c-de99f432ca83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.650520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1221/2448] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:53 2022] PUT /identity/v3/projects/90de1322f74d4967b44b0edfbc864ebd/users/2fc6e80da35949f394a257abf01fa8ca/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:53.696584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7da2bf42-f6fb-495e-b12c-de99f432ca83 None None] MFA Rules not processed for user `2fc6e80da35949f394a257abf01fa8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:53.697820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7da2bf42-f6fb-495e-b12c-de99f432ca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:53.817166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1228/2449] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:53.818077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.899245 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09893919-7608-417e-9c45-af3ec13e3228 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:53.909886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:53.910844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:53.912157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:53.912596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:53.913989 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:53.916164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:53.957011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1222/2450] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:53 2022] POST /identity/v3/projects => generated 385 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:53.961365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2bf42-f6fb-495e-b12c-de99f432ca83 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:53.978812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2bf42-f6fb-495e-b12c-de99f432ca83 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.063518 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.064381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.064741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.065092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.066175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.068273 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.095118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1229/2451] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:53 2022] POST /identity/v3/users => generated 293 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:54.099048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.110771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c1c360-03c1-4ac5-8d3b-bbf10b3e72b0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.116743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.118212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.118350 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.118607 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.119422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.121179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.133409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1223/2452] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:54.137228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.158308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d5131a5-97a3-47e0-a4db-97568cc3406f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.178025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1453ec0-da6a-40a1-b014-eb746adfea5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.178647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1453ec0-da6a-40a1-b014-eb746adfea5a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.178986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1453ec0-da6a-40a1-b014-eb746adfea5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.179269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1453ec0-da6a-40a1-b014-eb746adfea5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.180700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1453ec0-da6a-40a1-b014-eb746adfea5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.182497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1453ec0-da6a-40a1-b014-eb746adfea5a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.198996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1230/2453] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:54.200223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.210590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488446d-32f3-40e8-9e6c-6fa2ab41667c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.216788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.217461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.217745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.218312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] PATH_INFO: `/v3/projects/0cf77736b45e42fba69b4ffb2323bd69/users/4576ab8cf9734d048d5d035723a09f7f/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.234831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cf77736b45e42fba69b4ffb2323bd69, user_id=4576ab8cf9734d048d5d035723a09f7f, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.243961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.260084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1224/2454] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:54 2022] PUT /identity/v3/projects/0cf77736b45e42fba69b4ffb2323bd69/users/4576ab8cf9734d048d5d035723a09f7f/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:54.269414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daafb9d-7e25-43fc-b8dd-bd0cd923e0b2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.269622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daafb9d-7e25-43fc-b8dd-bd0cd923e0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.269910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daafb9d-7e25-43fc-b8dd-bd0cd923e0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.302728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:54.306950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1daafb9d-7e25-43fc-b8dd-bd0cd923e0b2 None None] MFA Rules not processed for user `4576ab8cf9734d048d5d035723a09f7f`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:54.309047 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1daafb9d-7e25-43fc-b8dd-bd0cd923e0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:54.383458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1231/2455] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:54.384382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.451068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4506fd8b-bae8-4ee5-b9ed-d2caab5af651 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.455781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.456438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.456717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.457022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.457671 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.459579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.489074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1225/2456] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:27:54 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:27:54.493532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daafb9d-7e25-43fc-b8dd-bd0cd923e0b2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.506813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daafb9d-7e25-43fc-b8dd-bd0cd923e0b2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.584155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.585135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.586873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.587359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.588288 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.591554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.617055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1232/2457] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:27:54 2022] POST /identity/v3/users => generated 299 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:27:54.618456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.628737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d7d4b9-17c5-4a61-875b-a569461ed698 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.633436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.634094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.634376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.634650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.635382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.636994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.642556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:54.650059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1226/2458] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:27:54.653026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.662816 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089695a2-262d-4b2e-8149-a3ac0692f632 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.667946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.668830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.669111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.669402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] PATH_INFO: `/v3/projects/0da74b03c72e4378a84f44ec6dbf2135/users/3b26eb6cf6fb4008b8424f5c781e2275/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.685232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da74b03c72e4378a84f44ec6dbf2135, user_id=3b26eb6cf6fb4008b8424f5c781e2275, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.688000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.702839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1233/2459] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:54 2022] PUT /identity/v3/projects/0da74b03c72e4378a84f44ec6dbf2135/users/3b26eb6cf6fb4008b8424f5c781e2275/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:27:54.707777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.722273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469869cd-14c8-4780-a5a6-6fd63f212af7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.809592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6811524c-1cac-4263-af25-82840affa7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.810307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811524c-1cac-4263-af25-82840affa7e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.810699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811524c-1cac-4263-af25-82840affa7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.810927 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6811524c-1cac-4263-af25-82840affa7e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.812707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811524c-1cac-4263-af25-82840affa7e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.816871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6811524c-1cac-4263-af25-82840affa7e3 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.837109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1227/2460] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:27:54 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:27:54.838286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:54.848822 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253254ca-aae7-49de-9e28-ca639f085a73 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:54.853567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:54.854457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.855387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.855387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] PATH_INFO: `/v3/projects/0da74b03c72e4378a84f44ec6dbf2135/users/3b26eb6cf6fb4008b8424f5c781e2275/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.857082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0da74b03c72e4378a84f44ec6dbf2135, user_id=3b26eb6cf6fb4008b8424f5c781e2275, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:54.859787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:54.879695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1234/2461] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:27:54 2022] PUT /identity/v3/projects/0da74b03c72e4378a84f44ec6dbf2135/users/3b26eb6cf6fb4008b8424f5c781e2275/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:27:54.891533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bcae6-7eaf-472a-850f-7d45ba54bf4a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:54.892074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bcae6-7eaf-472a-850f-7d45ba54bf4a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:54.892564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bcae6-7eaf-472a-850f-7d45ba54bf4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:54.926162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5f5bcae6-7eaf-472a-850f-7d45ba54bf4a None None] MFA Rules not processed for user `3b26eb6cf6fb4008b8424f5c781e2275`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:54.927549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5f5bcae6-7eaf-472a-850f-7d45ba54bf4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:55.035369 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1228/2462] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:27:55.044048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:55.045903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:55.074893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534bc79-8f45-48dc-90e7-e86122e77c38 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:55.075186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534bc79-8f45-48dc-90e7-e86122e77c38 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:55.075465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534bc79-8f45-48dc-90e7-e86122e77c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:55.107008 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4534bc79-8f45-48dc-90e7-e86122e77c38 None None] MFA Rules not processed for user `f28b6a7ebf8c4cbab9193b95b8de6818`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:27:55.108430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4534bc79-8f45-48dc-90e7-e86122e77c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:27:55.139495 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41478868-ae75-498b-bf0e-2594a46f42a8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:55.144314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:55.145579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:55.145871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:55.146152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:55.157248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:55.184210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1229/2463] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:27:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:27:55.284580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:55.286188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:55.307309 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1235/2464] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:27:57.045203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534bc79-8f45-48dc-90e7-e86122e77c38 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:57.057591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534bc79-8f45-48dc-90e7-e86122e77c38 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:57.062005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:57.062643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:57.062979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:57.063147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:57.097072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:27:57.141277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:57.143278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:57.153519 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1230/2465] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:27:57.965475 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:27:57.974633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a13383-716f-419e-8edd-70d6c2c4491e service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:27:57.979051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46528e27-2e4e-4132-bdd0-6a7efa497da0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:27:57.979864 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528e27-2e4e-4132-bdd0-6a7efa497da0 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:27:57.980226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528e27-2e4e-4132-bdd0-6a7efa497da0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:27:57.980579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46528e27-2e4e-4132-bdd0-6a7efa497da0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:27:57.992839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528e27-2e4e-4132-bdd0-6a7efa497da0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:27:57.994531 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46528e27-2e4e-4132-bdd0-6a7efa497da0 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:27:58.006490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1236/2466] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:27:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:28:02.180821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:02.264954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f5ba17-c57f-422a-9818-0c798e8a42b2 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:02.270104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:02.271261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:02.271844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:02.272165 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] PATH_INFO: `/v3/users/2fc6e80da35949f394a257abf01fa8ca` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:02.274317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc6e80da35949f394a257abf01fa8ca)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:02.276541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:02.339231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc6e80da35949f394a257abf01fa8ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:02.348634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc6e80da35949f394a257abf01fa8ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:02.350332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc6e80da35949f394a257abf01fa8ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:02.358387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc6e80da35949f394a257abf01fa8ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:02.376954 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc6e80da35949f394a257abf01fa8ca'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:02.387835 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1231/2467] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:28:02 2022] DELETE /identity/v3/users/2fc6e80da35949f394a257abf01fa8ca => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:02.390859 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:02.391321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:02.391435 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:02.414479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:02.415722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:02.487679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1237/2468] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:02.497328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:02.500620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:02.558572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:02.572100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bcbc6b-4e46-4bc6-a585-e2f458e18308 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:02.576198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:02.576952 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:02.577236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:02.577521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:02.588443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:02.590388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:02.602813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1232/2469] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:28:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:28:03.109181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:03.110645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:03.128804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd20f122-1a1b-4d4d-8e4e-44935b0a38e7 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:03.133699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:03.134722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:03.135050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:03.135219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] PATH_INFO: `/v3/projects/90de1322f74d4967b44b0edfbc864ebd` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:03.136548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90de1322f74d4967b44b0edfbc864ebd)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:03.138454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:03.188659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90de1322f74d4967b44b0edfbc864ebd', 'request_id': 'req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.197039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90de1322f74d4967b44b0edfbc864ebd', 'request_id': 'req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.203460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90de1322f74d4967b44b0edfbc864ebd', 'request_id': 'req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.213855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] The token cache is being invalidate because project 90de1322f74d4967b44b0edfbc864ebd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:28:03.214146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.214778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.216349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1238/2470] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:28:03 2022] DELETE /identity/v3/projects/90de1322f74d4967b44b0edfbc864ebd => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:03.220426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:03.230547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4a4f61-2623-474d-9ceb-2e40176491ef service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:03.285251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:03.286131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:03.286413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:03.286693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] PATH_INFO: `/v3/users/4576ab8cf9734d048d5d035723a09f7f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:03.287915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4576ab8cf9734d048d5d035723a09f7f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:03.290348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:03.339318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4576ab8cf9734d048d5d035723a09f7f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.345734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4576ab8cf9734d048d5d035723a09f7f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.346499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4576ab8cf9734d048d5d035723a09f7f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.353798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4576ab8cf9734d048d5d035723a09f7f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.365396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4576ab8cf9734d048d5d035723a09f7f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:03.371889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1233/2471] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:28:03 2022] DELETE /identity/v3/users/4576ab8cf9734d048d5d035723a09f7f => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:03.711679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:03.714107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:03.757444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b308c-5b30-4d17-a36a-3f647cf7fb33 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:03.795854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:03.888170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:03.896926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:03.897301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:03.897609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] PATH_INFO: `/v3/projects/0cf77736b45e42fba69b4ffb2323bd69` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:03.900252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cf77736b45e42fba69b4ffb2323bd69)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:03.908306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:04.012615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cf77736b45e42fba69b4ffb2323bd69', 'request_id': 'req-0d7bab33-e69f-4aeb-964d-f8f38ab96906'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.030521 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cf77736b45e42fba69b4ffb2323bd69', 'request_id': 'req-0d7bab33-e69f-4aeb-964d-f8f38ab96906'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.059579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cf77736b45e42fba69b4ffb2323bd69', 'request_id': 'req-0d7bab33-e69f-4aeb-964d-f8f38ab96906'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.085573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] The token cache is being invalidate because project 0cf77736b45e42fba69b4ffb2323bd69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:28:04.086130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.086876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.091784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1239/2472] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:28:03 2022] DELETE /identity/v3/projects/0cf77736b45e42fba69b4ffb2323bd69 => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:04.094343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:04.096722 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:04.132930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04aca518-319b-418c-aed1-ad0077d21607 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:04.220679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:04.221920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:04.222443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:04.222914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] PATH_INFO: `/v3/users/3b26eb6cf6fb4008b8424f5c781e2275` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:04.224494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b26eb6cf6fb4008b8424f5c781e2275)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:04.227389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:04.235707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:04.293742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b26eb6cf6fb4008b8424f5c781e2275'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.307951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b26eb6cf6fb4008b8424f5c781e2275'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.308741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b26eb6cf6fb4008b8424f5c781e2275'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.317585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b26eb6cf6fb4008b8424f5c781e2275'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.337293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-46c2bba4-7aab-4d7a-ad6f-b7585b4d80a2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b26eb6cf6fb4008b8424f5c781e2275'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.348299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1234/2473] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:28:04 2022] DELETE /identity/v3/users/3b26eb6cf6fb4008b8424f5c781e2275 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:04.780758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:04.782363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:04.797693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7bab33-e69f-4aeb-964d-f8f38ab96906 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:04.878635 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:04.879925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:04.880281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:04.880746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] PATH_INFO: `/v3/projects/0da74b03c72e4378a84f44ec6dbf2135` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:04.882414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0da74b03c72e4378a84f44ec6dbf2135)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:04.885411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:04.954910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0da74b03c72e4378a84f44ec6dbf2135', 'request_id': 'req-a7035f37-1805-4ad8-8d1e-b58e2e658966'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.963080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:04.971337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0da74b03c72e4378a84f44ec6dbf2135', 'request_id': 'req-a7035f37-1805-4ad8-8d1e-b58e2e658966'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:04.986646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0da74b03c72e4378a84f44ec6dbf2135', 'request_id': 'req-a7035f37-1805-4ad8-8d1e-b58e2e658966'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:05.015033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] The token cache is being invalidate because project 0da74b03c72e4378a84f44ec6dbf2135 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:28:05.015169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:05.015974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a7035f37-1805-4ad8-8d1e-b58e2e658966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:05.019781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1240/2474] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:28:04 2022] DELETE /identity/v3/projects/0da74b03c72e4378a84f44ec6dbf2135 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:05.023189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021bdf9-21b7-4f79-91b6-ca9a297f1a21 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.023592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021bdf9-21b7-4f79-91b6-ca9a297f1a21 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.024000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021bdf9-21b7-4f79-91b6-ca9a297f1a21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.044435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f021bdf9-21b7-4f79-91b6-ca9a297f1a21 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:05.045667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f021bdf9-21b7-4f79-91b6-ca9a297f1a21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:05.119796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1235/2475] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:05.182790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6cf9e-e5d8-45e5-a7e1-5c6f02451dea None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.183093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6cf9e-e5d8-45e5-a7e1-5c6f02451dea None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.183371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d6cf9e-e5d8-45e5-a7e1-5c6f02451dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.204151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-70d6cf9e-e5d8-45e5-a7e1-5c6f02451dea None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:05.205506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70d6cf9e-e5d8-45e5-a7e1-5c6f02451dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:05.213605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1241/2476] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:05.217923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021bdf9-21b7-4f79-91b6-ca9a297f1a21 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:05.228710 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021bdf9-21b7-4f79-91b6-ca9a297f1a21 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:05.233281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:05.233940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.234223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.234501 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.235222 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:28:05.235707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:05.237346 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:05.249796 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1236/2477] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:28:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:28:05.254177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6cf9e-e5d8-45e5-a7e1-5c6f02451dea None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:05.266808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d6cf9e-e5d8-45e5-a7e1-5c6f02451dea None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:05.271596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:05.272294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.272578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.272858 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.273553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:05.275446 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:05.305157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1242/2478] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:28:05 2022] POST /identity/v3/projects => generated 371 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:28:05.309025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:05.321035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e9b1f-b86d-4ac3-824e-abe4b3f691ae admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:05.384364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:05.385179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.385531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.385871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.386642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:05.388728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:05.418565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1237/2479] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:28:05 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:28:05.418991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:05.441447 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd05bd9-319d-4647-9880-6426333b01cf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:05.447687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:05.448476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.448857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.449200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.450072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:05.452292 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:05.465388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1243/2480] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:28:05.468615 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:05.475755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:05.484917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83186100-6616-452c-8bb2-1bb579e5c30c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:05.491760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc6d536-79ec-4953-9a8a-2f626ad4db28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:05.493213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d536-79ec-4953-9a8a-2f626ad4db28 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.493537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d536-79ec-4953-9a8a-2f626ad4db28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.493895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc6d536-79ec-4953-9a8a-2f626ad4db28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.495108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6d536-79ec-4953-9a8a-2f626ad4db28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:05.497484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc6d536-79ec-4953-9a8a-2f626ad4db28 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:05.526055 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1238/2481] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:28:05 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:28:05.530134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:05.542877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c606bdd-c16b-4780-9d1a-d2ac203b3684 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:05.550300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:05.551263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.552099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.552389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] PATH_INFO: `/v3/projects/21dabce5b3274cfcba5840982b59434e/users/d8c69ecf2f23490c92ab315433273817/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.576107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21dabce5b3274cfcba5840982b59434e, user_id=d8c69ecf2f23490c92ab315433273817, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:05.579064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:05.595222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1244/2482] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:28:05 2022] PUT /identity/v3/projects/21dabce5b3274cfcba5840982b59434e/users/d8c69ecf2f23490c92ab315433273817/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:28:05.602003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722d97-cc16-427c-b92b-bef155d694fe None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.602395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722d97-cc16-427c-b92b-bef155d694fe None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.602748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3722d97-cc16-427c-b92b-bef155d694fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.627924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a3722d97-cc16-427c-b92b-bef155d694fe None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:05.627924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a3722d97-cc16-427c-b92b-bef155d694fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:05.721577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1239/2483] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:05.732308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:05.735706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:05.800454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423a1685-f345-4f88-af34-8953916f0353 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:05.805220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4309126-5f71-471a-b58e-9696fdcb09f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:05.805911 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309126-5f71-471a-b58e-9696fdcb09f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.806364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309126-5f71-471a-b58e-9696fdcb09f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.806645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4309126-5f71-471a-b58e-9696fdcb09f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.816450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309126-5f71-471a-b58e-9696fdcb09f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:05.817925 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4309126-5f71-471a-b58e-9696fdcb09f7 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:05.828458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1245/2484] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:28:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:28:05.936011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f0d2f-d674-44d9-baa5-31acf757f1c7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:05.936365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f0d2f-d674-44d9-baa5-31acf757f1c7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:05.936939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3f0d2f-d674-44d9-baa5-31acf757f1c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:05.957669 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7d3f0d2f-d674-44d9-baa5-31acf757f1c7 None None] MFA Rules not processed for user `92b758b22f054dd991e8af99354a134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:05.959529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7d3f0d2f-d674-44d9-baa5-31acf757f1c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:06.033943 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1240/2485] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:11.744564 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa616c5-a058-4ae4-8908-e07c19988c2a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:11.745106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa616c5-a058-4ae4-8908-e07c19988c2a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:11.745106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa616c5-a058-4ae4-8908-e07c19988c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:11.773830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4fa616c5-a058-4ae4-8908-e07c19988c2a None None] MFA Rules not processed for user `d8c69ecf2f23490c92ab315433273817`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:11.775087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4fa616c5-a058-4ae4-8908-e07c19988c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:11.842824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1246/2486] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:11.847302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f0d2f-d674-44d9-baa5-31acf757f1c7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:11.859335 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3f0d2f-d674-44d9-baa5-31acf757f1c7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:11.863925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:11.864583 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:11.864871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:11.865152 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:11.865854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:11.867832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:11.894010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1241/2487] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:28:11 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:28:11.898675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa616c5-a058-4ae4-8908-e07c19988c2a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:11.911171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa616c5-a058-4ae4-8908-e07c19988c2a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:11.919516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:12.003009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:12.008327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:12.008873 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:12.009219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:12.010368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:12.014949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:12.048117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1247/2488] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:28:11 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:28:12.052510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:12.065125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d32476-fe94-40b8-9aa4-eccb13c8febd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:12.070815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:12.071483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:12.071783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:12.072064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:12.072790 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:12.074403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:12.093883 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1242/2489] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:28:12.094617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:12.111831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e2bd9e-07a3-42c5-bfcc-9643ca030c68 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:12.117785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:12.118824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:12.119212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:12.120342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] PATH_INFO: `/v3/projects/7a764964d68c403b954db4aa89a05947/users/ac0519a709744125a2dad4d45bcc922b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:12.141819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a764964d68c403b954db4aa89a05947, user_id=ac0519a709744125a2dad4d45bcc922b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:12.144989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:12.173306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1248/2490] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:28:12 2022] PUT /identity/v3/projects/7a764964d68c403b954db4aa89a05947/users/ac0519a709744125a2dad4d45bcc922b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:28:12.174844 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:12.190667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca1fdf7-43b2-4c26-8e42-91a42752e099 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:12.311489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588766a5-c634-440e-9651-94b26a5b31c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:12.314664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588766a5-c634-440e-9651-94b26a5b31c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:12.314664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588766a5-c634-440e-9651-94b26a5b31c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:12.314664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588766a5-c634-440e-9651-94b26a5b31c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:12.316610 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588766a5-c634-440e-9651-94b26a5b31c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:12.318707 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588766a5-c634-440e-9651-94b26a5b31c4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:12.331434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:12.345525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1243/2491] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:28:12 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:28:12.345663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:12.357580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef8a6d-7e00-4091-a927-5111e6c5bd0d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:12.370216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:12.373376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:12.373674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:12.373960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] PATH_INFO: `/v3/projects/7a764964d68c403b954db4aa89a05947/users/ac0519a709744125a2dad4d45bcc922b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:12.380732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a764964d68c403b954db4aa89a05947, user_id=ac0519a709744125a2dad4d45bcc922b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:12.385510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:12.403114 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1249/2492] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:28:12 2022] PUT /identity/v3/projects/7a764964d68c403b954db4aa89a05947/users/ac0519a709744125a2dad4d45bcc922b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:14.806978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c46ae-5544-49ec-b90e-016a5536e0d7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:14.808081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c46ae-5544-49ec-b90e-016a5536e0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:14.808704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714c46ae-5544-49ec-b90e-016a5536e0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:14.847809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-714c46ae-5544-49ec-b90e-016a5536e0d7 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:14.849184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-714c46ae-5544-49ec-b90e-016a5536e0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:14.916787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1244/2493] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:28:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:14.941982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:14.949259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:15.105167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17dd024c-5cc5-4914-99cb-85d4ea118c86 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:15.119103 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:15.119903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:15.120692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:15.120692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:15.131513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:15.140404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:15.150537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1250/2494] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:28:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:28:16.093492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb446b6-8fc3-425a-9050-789b05052c34 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:16.094431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb446b6-8fc3-425a-9050-789b05052c34 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:16.094882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb446b6-8fc3-425a-9050-789b05052c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:16.136666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2fb446b6-8fc3-425a-9050-789b05052c34 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:16.139244 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2fb446b6-8fc3-425a-9050-789b05052c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:16.166560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1245/2495] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jul 7 10:28:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:16.166904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:16.181195 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4a959-6447-4aa4-9502-3f773425f4fc service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:16.188347 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:16.189524 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:16.190010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:16.190509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:16.252917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:16.254662 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:16.265409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1251/2496] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 07 10:28:16.789428 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227e6c3-6caf-48ca-a30b-b1ce0cffc38c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:16.790024 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227e6c3-6caf-48ca-a30b-b1ce0cffc38c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:16.790502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3227e6c3-6caf-48ca-a30b-b1ce0cffc38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:16.814935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3227e6c3-6caf-48ca-a30b-b1ce0cffc38c None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:16.817069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3227e6c3-6caf-48ca-a30b-b1ce0cffc38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:16.828775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1246/2497] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:28:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:16.841871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:16.857725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3d925b-7e7d-4ced-ad2a-624c21a19b23 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:16.869774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:16.870745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:16.871177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:16.871602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:16.881931 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:16.884197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:16.922525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1252/2498] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:28:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:28:17.499569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:17.500427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:17.500910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:17.541061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:17.543737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:17.556356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1247/2499] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jul 7 10:28:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:17.572655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:17.588918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c03988-d681-4120-80a0-ba471a112997 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:17.610596 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f129dd68-f2da-4a81-903a-7bb149e8f5a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:17.611423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129dd68-f2da-4a81-903a-7bb149e8f5a2 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:17.611802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129dd68-f2da-4a81-903a-7bb149e8f5a2 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:17.612157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f129dd68-f2da-4a81-903a-7bb149e8f5a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:17.642107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129dd68-f2da-4a81-903a-7bb149e8f5a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:17.644167 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f129dd68-f2da-4a81-903a-7bb149e8f5a2 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:17.663022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1253/2500] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:28:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:28:20.336902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:20.340773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:20.663884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd746c2-ff13-4116-a8c3-bc856e8e7ff7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:20.681330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:20.683271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:20.683561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:20.683863 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] PATH_INFO: `/v3/users/ee9bcb3226f240a5951ab9971f6bb654` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:20.688616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9bcb3226f240a5951ab9971f6bb654)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:20.690588 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:20.762218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:20.817595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9bcb3226f240a5951ab9971f6bb654'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:20.831651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:20.832161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:20.832395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:20.847157 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9bcb3226f240a5951ab9971f6bb654'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:20.847850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9bcb3226f240a5951ab9971f6bb654'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:20.869430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9bcb3226f240a5951ab9971f6bb654'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:20.892045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:20.903942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6922763d-24e6-4320-9c99-92b11fda2b8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9bcb3226f240a5951ab9971f6bb654'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:20.916865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1248/2501] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:28:20 2022] DELETE /identity/v3/users/ee9bcb3226f240a5951ab9971f6bb654 => generated 0 bytes in 586 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:28:20.918595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:20.918887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:20.919167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:20.923851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] MFA Rules not processed for user `ac0519a709744125a2dad4d45bcc922b`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:20.925204 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:20.982300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:20.988438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:21.086715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1254/2502] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:21.087989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:21.094629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:21.203544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1249/2503] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:28:21.253255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:21.254972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:21.301237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecb5bcd-208f-40b5-939f-ff7308ab927a None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:21.306140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:21.307144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:21.307574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:21.308025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:21.311054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0dd07a-075c-4484-a2eb-03aae1ea9b4f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:21.322674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:21.323573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:21.323988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:21.324272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:21.324627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:21.331882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:21.335092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:21.357613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1250/2504] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:28:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:28:21.390339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:21.392545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:21.431880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:21.433335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:21.462747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1255/2505] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:28:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:28:21.569561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c63762-bea4-4e23-8f38-16bad9119b89 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:21.582233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b849d-f565-4b2b-b13b-ad7f807881f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:21.586894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b849d-f565-4b2b-b13b-ad7f807881f9 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:21.587316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b849d-f565-4b2b-b13b-ad7f807881f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:21.587603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b849d-f565-4b2b-b13b-ad7f807881f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:21.590631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d86b849d-f565-4b2b-b13b-ad7f807881f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:21.616069 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:21.712941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b849d-f565-4b2b-b13b-ad7f807881f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:21.715433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b849d-f565-4b2b-b13b-ad7f807881f9 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:21.744438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1251/2506] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:28:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:28:21.864561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:21.866367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:21.899718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19dd98-81bb-4993-a2d2-7a1a5b5612ab service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:21.913427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:21.915026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:21.915382 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:21.916139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] PATH_INFO: `/v3/projects/71b2e4cb235e41da9ade19672bdd2e4d` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:21.918551 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b2e4cb235e41da9ade19672bdd2e4d)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:21.922109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:21.932611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:22.019492 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b2e4cb235e41da9ade19672bdd2e4d', 'request_id': 'req-b57ce0ec-6df5-4f03-9334-ae804a478af0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:22.033035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b2e4cb235e41da9ade19672bdd2e4d', 'request_id': 'req-b57ce0ec-6df5-4f03-9334-ae804a478af0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:22.051052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b2e4cb235e41da9ade19672bdd2e4d', 'request_id': 'req-b57ce0ec-6df5-4f03-9334-ae804a478af0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:22.065840 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] The token cache is being invalidate because project 71b2e4cb235e41da9ade19672bdd2e4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:28:22.066139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:22.066809 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b57ce0ec-6df5-4f03-9334-ae804a478af0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:28:22.079832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1256/2507] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:28:21 2022] DELETE /identity/v3/projects/71b2e4cb235e41da9ade19672bdd2e4d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:28:24.504896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:24.505372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:24.505633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:24.552207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] MFA Rules not processed for user `d8c69ecf2f23490c92ab315433273817`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:24.554366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:24.684780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1252/2508] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:28:24 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:32.062368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=22,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:34.654041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=47 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:28:42.769361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:42.770162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:42.770162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:42.799778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:28:42.801207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:42.865761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1257/2509] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:28:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:28:42.889686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:42.891090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:42.962903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e22b4b-4118-4e2b-bd2f-ab7b33c26d20 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:42.968362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9884f432-c5b8-4577-94c0-9f54569381cf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:42.969014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884f432-c5b8-4577-94c0-9f54569381cf service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:42.969295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884f432-c5b8-4577-94c0-9f54569381cf service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:42.969566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9884f432-c5b8-4577-94c0-9f54569381cf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:42.978223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884f432-c5b8-4577-94c0-9f54569381cf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:42.979843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9884f432-c5b8-4577-94c0-9f54569381cf service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:42.987792 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1253/2510] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:28:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:28:51.625667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:28:51.627119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:51.756136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda4e296-7fe3-4a2d-bd71-75fe041aae6a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:28:51.765783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:28:51.767053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:28:51.767053 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:28:51.767206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:28:51.769458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:28:51.785399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:28:51.789064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:28:51.801700 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1258/2511] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:28:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:28:52.984582 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=32 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:01.795960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=73 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:05.366760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:05.367258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:05.367352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:05.384413 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:05.385593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:05.392645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1254/2512] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jul 7 10:29:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:05.404771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:05.416533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e2e4ca-0360-4501-b345-ad8c7bfd8a0a service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:05.421587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:05.422232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:05.422549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:05.422833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:05.431904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:05.433487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:05.441546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1259/2513] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:29:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:29:10.611617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:10.613726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:10.682521 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8927d1a-bb53-4ccc-ad21-65e0b56a2b96 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:10.686988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:10.687953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:10.688236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:10.688520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] PATH_INFO: `/v3/users/f28b6a7ebf8c4cbab9193b95b8de6818` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:10.689747 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f28b6a7ebf8c4cbab9193b95b8de6818)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:10.691655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:10.735627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f28b6a7ebf8c4cbab9193b95b8de6818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:10.743721 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f28b6a7ebf8c4cbab9193b95b8de6818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:10.744356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f28b6a7ebf8c4cbab9193b95b8de6818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:10.751090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f28b6a7ebf8c4cbab9193b95b8de6818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:10.762770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-28282919-21aa-4a25-a136-13cdfb2f22a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f28b6a7ebf8c4cbab9193b95b8de6818'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:10.768755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1255/2514] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:29:10 2022] DELETE /identity/v3/users/f28b6a7ebf8c4cbab9193b95b8de6818 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:29:10.943228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:10.955528 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddbf32c-4902-4c86-9471-67a7f30bd4bb service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.012107 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.012975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.013275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.013538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] PATH_INFO: `/v3/projects/3d3b58b3beed423aba9a7c28d1e09c88` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.014734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d3b58b3beed423aba9a7c28d1e09c88)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.016717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.064775 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d3b58b3beed423aba9a7c28d1e09c88', 'request_id': 'req-a59391d9-7a4b-4c38-8f62-863ff5592fad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:11.072391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d3b58b3beed423aba9a7c28d1e09c88', 'request_id': 'req-a59391d9-7a4b-4c38-8f62-863ff5592fad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:11.078451 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d3b58b3beed423aba9a7c28d1e09c88', 'request_id': 'req-a59391d9-7a4b-4c38-8f62-863ff5592fad'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:11.089203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] The token cache is being invalidate because project 3d3b58b3beed423aba9a7c28d1e09c88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:29:11.089489 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:11.090115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a59391d9-7a4b-4c38-8f62-863ff5592fad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:11.091640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1260/2515] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:29:10 2022] DELETE /identity/v3/projects/3d3b58b3beed423aba9a7c28d1e09c88 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:29:11.096612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ac7a1-709b-4c89-9071-c85e21cee8d4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.096898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ac7a1-709b-4c89-9071-c85e21cee8d4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.097175 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52ac7a1-709b-4c89-9071-c85e21cee8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.143117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b52ac7a1-709b-4c89-9071-c85e21cee8d4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:11.144266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b52ac7a1-709b-4c89-9071-c85e21cee8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:11.187125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:11.203928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1256/2516] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:11.212008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfcf3e-e64a-43c3-b7de-1bc5f093ab9d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.212300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfcf3e-e64a-43c3-b7de-1bc5f093ab9d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.212580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcfcf3e-e64a-43c3-b7de-1bc5f093ab9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.230505 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-2fcfcf3e-e64a-43c3-b7de-1bc5f093ab9d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:11.231720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-2fcfcf3e-e64a-43c3-b7de-1bc5f093ab9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:11.239578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1261/2517] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:11.243715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ac7a1-709b-4c89-9071-c85e21cee8d4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:11.253473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52ac7a1-709b-4c89-9071-c85e21cee8d4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.258476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.259121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.259400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.259677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.260425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:29:11.260736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.262342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.276654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1257/2518] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:29:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:29:11.278933 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfcf3e-e64a-43c3-b7de-1bc5f093ab9d None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:11.288183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcfcf3e-e64a-43c3-b7de-1bc5f093ab9d None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.292802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.293454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.293744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.294025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.294717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.296647 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.322863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1262/2519] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:29:11 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:29:11.327005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:11.337527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0a32-9a9f-4f41-9e8c-98898f48ecdd admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.393058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.393773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.394005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.394292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.395017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.397573 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.417063 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1258/2520] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:29:11 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:29:11.421479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:11.431768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8098486-367f-4145-9f61-a63b96e68256 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.436314 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.436975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.437262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.437541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.438267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.439898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.450899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1263/2521] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:29:11.455052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:11.465652 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af188cd9-c860-4ddd-926f-e73e5db666c4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.470230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215a0b4e-98a1-41f7-b982-decaac308290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.470877 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a0b4e-98a1-41f7-b982-decaac308290 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.471167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a0b4e-98a1-41f7-b982-decaac308290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.471448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215a0b4e-98a1-41f7-b982-decaac308290 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.472205 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a0b4e-98a1-41f7-b982-decaac308290 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.473798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215a0b4e-98a1-41f7-b982-decaac308290 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.483457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1259/2522] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:29:11.487241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:11.497087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25acb69f-92dc-403f-9658-017843e81521 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.501936 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.502831 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.503117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.503395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] PATH_INFO: `/v3/projects/2ba6d6d61f4b4f71a39a97f88444b628/users/1b8e077ba14848fb864b798a912474a0/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.519948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ba6d6d61f4b4f71a39a97f88444b628, user_id=1b8e077ba14848fb864b798a912474a0, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.522945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.536501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1264/2523] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:29:11 2022] PUT /identity/v3/projects/2ba6d6d61f4b4f71a39a97f88444b628/users/1b8e077ba14848fb864b798a912474a0/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:29:11.547259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497789d-116b-4cca-9a4e-3a7793029080 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.547547 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497789d-116b-4cca-9a4e-3a7793029080 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.547866 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0497789d-116b-4cca-9a4e-3a7793029080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.576388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0497789d-116b-4cca-9a4e-3a7793029080 None None] MFA Rules not processed for user `1b8e077ba14848fb864b798a912474a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:11.577612 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0497789d-116b-4cca-9a4e-3a7793029080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:11.643235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1260/2524] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:11.654454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:11.656260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:11.673988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:11.738480 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54b3fe0-c873-4051-ab3c-09ee948b6659 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:11.742722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf302a7-b630-4a48-9dfd-6897fbb04d5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:11.743377 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf302a7-b630-4a48-9dfd-6897fbb04d5b service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:11.743666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf302a7-b630-4a48-9dfd-6897fbb04d5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:11.743984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf302a7-b630-4a48-9dfd-6897fbb04d5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:11.752554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf302a7-b630-4a48-9dfd-6897fbb04d5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:11.753990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf302a7-b630-4a48-9dfd-6897fbb04d5b service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:11.763504 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1265/2525] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:29:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:29:15.342964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497789d-116b-4cca-9a4e-3a7793029080 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:15.390004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:15.418657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0497789d-116b-4cca-9a4e-3a7793029080 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:15.423568 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:15.424536 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:15.424890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:15.425169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] PATH_INFO: `/v3/users/1b8e077ba14848fb864b798a912474a0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:15.426458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b8e077ba14848fb864b798a912474a0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:15.428635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:15.474434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b8e077ba14848fb864b798a912474a0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:15.482511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b8e077ba14848fb864b798a912474a0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:15.483139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b8e077ba14848fb864b798a912474a0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:15.489938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b8e077ba14848fb864b798a912474a0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:15.501574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b8e077ba14848fb864b798a912474a0'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:15.510766 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1261/2526] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:29:15 2022] DELETE /identity/v3/users/1b8e077ba14848fb864b798a912474a0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:29:15.512317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:15.512729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:15.512897 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:15.531369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:15.532619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:15.589886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1266/2527] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:29:15.599322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:15.601451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:15.658182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22408aba-7f7c-4117-a299-400703946e9c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:15.661881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a900bd-4fec-4d2a-8762-361180c9c145 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:15.663137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a900bd-4fec-4d2a-8762-361180c9c145 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:15.663425 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a900bd-4fec-4d2a-8762-361180c9c145 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:15.663704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a900bd-4fec-4d2a-8762-361180c9c145 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:15.672746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a900bd-4fec-4d2a-8762-361180c9c145 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:15.674173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a900bd-4fec-4d2a-8762-361180c9c145 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:15.683576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1262/2528] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:29:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:29:15.944685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:15.946476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:15.958987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7debe0-78eb-4920-bf71-b7528c4586b5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:15.964342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:15.965291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:15.965692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:15.965962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] PATH_INFO: `/v3/projects/2ba6d6d61f4b4f71a39a97f88444b628` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:15.967149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ba6d6d61f4b4f71a39a97f88444b628)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:15.969119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.017131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ba6d6d61f4b4f71a39a97f88444b628', 'request_id': 'req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:16.025972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ba6d6d61f4b4f71a39a97f88444b628', 'request_id': 'req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:16.032422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ba6d6d61f4b4f71a39a97f88444b628', 'request_id': 'req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:16.043021 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] The token cache is being invalidate because project 2ba6d6d61f4b4f71a39a97f88444b628 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:29:16.043311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:16.044379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4e26936d-0140-46f2-ba4e-7cd0f07cc8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:29:16.045663 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1267/2529] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:29:15 2022] DELETE /identity/v3/projects/2ba6d6d61f4b4f71a39a97f88444b628 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:29:16.050578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2737cc-07d5-49ff-b246-84f63cff222e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.050940 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2737cc-07d5-49ff-b246-84f63cff222e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.051283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2737cc-07d5-49ff-b246-84f63cff222e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.069808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9a2737cc-07d5-49ff-b246-84f63cff222e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:16.071249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9a2737cc-07d5-49ff-b246-84f63cff222e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:16.134390 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1263/2530] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:29:16.202597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f12867-a6e8-47e2-8944-91732861e84e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.202887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f12867-a6e8-47e2-8944-91732861e84e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.203169 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f12867-a6e8-47e2-8944-91732861e84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.224941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-b0f12867-a6e8-47e2-8944-91732861e84e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:16.226398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b0f12867-a6e8-47e2-8944-91732861e84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:16.235569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1268/2531] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:16.239783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2737cc-07d5-49ff-b246-84f63cff222e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:16.257406 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2737cc-07d5-49ff-b246-84f63cff222e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:16.265031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:16.266389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.266858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.267301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.268894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:29:16.269749 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:16.274080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.276260 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:16.295787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1264/2532] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:29:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:29:16.298994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f12867-a6e8-47e2-8944-91732861e84e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:16.312581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f12867-a6e8-47e2-8944-91732861e84e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:16.321057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:16.323416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.323850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.324232 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.325040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:16.327067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.342856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:16.371871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1269/2533] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:29:16 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:29:16.371992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:16.385572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d597a706-3fb1-4b5e-a7c2-f63882e12eab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:16.449843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:16.450561 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.450845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.451126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.451850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:16.453818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.475782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1265/2534] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:29:16 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:29:16.479871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:16.491269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4398981-1aa6-45a0-8e7a-f02b4e4cab63 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:16.496275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:16.497000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.497283 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.497561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.498291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:16.500009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.514467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1270/2535] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:29:16.515770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:16.527808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2997-25e1-426e-8a2f-633be8b583a4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:16.533280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858cf090-ae10-4ccc-bb18-d711cd4e7d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:16.534045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cf090-ae10-4ccc-bb18-d711cd4e7d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.534328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cf090-ae10-4ccc-bb18-d711cd4e7d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.534604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858cf090-ae10-4ccc-bb18-d711cd4e7d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.535376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cf090-ae10-4ccc-bb18-d711cd4e7d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:16.537233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858cf090-ae10-4ccc-bb18-d711cd4e7d81 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.551782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1266/2536] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:29:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:29:16.554510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:16.566359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b01cfc-672f-4e9b-9625-82715ff2cfde admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:16.571529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:16.572493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.572764 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.573039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] PATH_INFO: `/v3/projects/12a91d56e163442dacf4c5ce7b28a132/users/059a87a80fc547db8aeb3bb03003cd2c/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.594327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12a91d56e163442dacf4c5ce7b28a132, user_id=059a87a80fc547db8aeb3bb03003cd2c, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:16.597468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.620049 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1271/2537] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:29:16 2022] PUT /identity/v3/projects/12a91d56e163442dacf4c5ce7b28a132/users/059a87a80fc547db8aeb3bb03003cd2c/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:29:16.625239 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51bbb3-23a3-4849-bb3b-2c08d8459c88 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.625659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51bbb3-23a3-4849-bb3b-2c08d8459c88 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.626079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba51bbb3-23a3-4849-bb3b-2c08d8459c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.654576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ba51bbb3-23a3-4849-bb3b-2c08d8459c88 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:16.656845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ba51bbb3-23a3-4849-bb3b-2c08d8459c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:16.763936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1267/2538] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:16.770389 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:16.772679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:16.835705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c958b61-c2e8-4b09-b9ed-f12d86d41008 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:16.839558 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:16.840626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:16.840976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:16.841252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:16.851312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:16.852814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:16.862789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1272/2539] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:29:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:29:21.231499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a40f3-f505-474c-a2f0-c08a9afa3f79 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:21.231942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a40f3-f505-474c-a2f0-c08a9afa3f79 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:21.232078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467a40f3-f505-474c-a2f0-c08a9afa3f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:21.300988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-467a40f3-f505-474c-a2f0-c08a9afa3f79 None None] MFA Rules not processed for user `059a87a80fc547db8aeb3bb03003cd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:21.302424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-467a40f3-f505-474c-a2f0-c08a9afa3f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:21.383733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1268/2540] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:21.388278 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:21.400498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c068b5-25b2-4be0-9813-e83515e9f44f service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:21.408090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:21.408830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:21.409115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:21.409398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:21.410145 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:21.413011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:21.443587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1273/2541] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:29:21 2022] POST /identity/v3/projects => generated 377 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:29:21.444681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a40f3-f505-474c-a2f0-c08a9afa3f79 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:21.450555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:21.457677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467a40f3-f505-474c-a2f0-c08a9afa3f79 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:21.517773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:21.518410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:21.518694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:21.519104 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:21.519833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:21.522309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:21.543282 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1269/2542] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:29:21 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:29:21.547257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:21.567042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc63bc8-d817-4625-a498-6082a994e4b7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:21.572649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:21.573327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:21.573620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:21.573904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:21.574750 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:21.576445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:21.588872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1274/2543] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:29:21.593228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:21.607571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a908afd-90c4-4063-a7b3-77f9146e8227 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:21.613178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:21.614123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:21.614468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:21.614753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] PATH_INFO: `/v3/projects/79d3d594053f4e62a2718c7ee83b8439/users/49b589a8aea6457d8a53d811d9b6fd16/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:21.632050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d3d594053f4e62a2718c7ee83b8439, user_id=49b589a8aea6457d8a53d811d9b6fd16, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:21.634885 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:21.650043 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1270/2544] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:29:21 2022] PUT /identity/v3/projects/79d3d594053f4e62a2718c7ee83b8439/users/49b589a8aea6457d8a53d811d9b6fd16/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:29:21.654018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:21.670112 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b69b8-22ae-468f-a72e-3cd5d9b0ebaa admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:21.764615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:21.765297 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:21.765586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:21.765866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:21.766628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:21.773629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:21.784325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1275/2545] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:29:21.788557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:21.799873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b83dd2-7fd3-4565-9610-7deebb6c2863 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:21.804810 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6695ca7d-3b27-4f9d-8e36-5cfd41e1f340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:21.805711 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695ca7d-3b27-4f9d-8e36-5cfd41e1f340 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:21.805994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695ca7d-3b27-4f9d-8e36-5cfd41e1f340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:21.806274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6695ca7d-3b27-4f9d-8e36-5cfd41e1f340 admin admin] PATH_INFO: `/v3/projects/79d3d594053f4e62a2718c7ee83b8439/users/49b589a8aea6457d8a53d811d9b6fd16/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:21.808341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695ca7d-3b27-4f9d-8e36-5cfd41e1f340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=79d3d594053f4e62a2718c7ee83b8439, user_id=49b589a8aea6457d8a53d811d9b6fd16, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:21.811028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6695ca7d-3b27-4f9d-8e36-5cfd41e1f340 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:21.822795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1271/2546] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:29:21 2022] PUT /identity/v3/projects/79d3d594053f4e62a2718c7ee83b8439/users/49b589a8aea6457d8a53d811d9b6fd16/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:29:22.796729 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:22.832795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:22.875781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f24c6dd-1012-4567-b44d-840f84f0d41a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:22.940132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:22.940791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:22.941077 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:22.941704 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:22.946797 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:23.032381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:23.034450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:23.050404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1276/2547] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:29:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:29:27.895731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8538c9c-9a00-46e9-8828-00a5bcd62a3c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:27.896400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8538c9c-9a00-46e9-8828-00a5bcd62a3c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:27.896400 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8538c9c-9a00-46e9-8828-00a5bcd62a3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:27.925154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f8538c9c-9a00-46e9-8828-00a5bcd62a3c None None] MFA Rules not processed for user `49b589a8aea6457d8a53d811d9b6fd16`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:27.926294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f8538c9c-9a00-46e9-8828-00a5bcd62a3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:27.997255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1272/2548] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:28.012241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:29:28.013609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:28.133709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e67210-ab86-4ecd-9f0e-32479a192520 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:29:28.139488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a9fc0-5228-4d90-a1ae-686178b690f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:29:28.140221 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9fc0-5228-4d90-a1ae-686178b690f9 service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:28.140506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9fc0-5228-4d90-a1ae-686178b690f9 service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:28.140779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a9fc0-5228-4d90-a1ae-686178b690f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:28.145241 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:28.200611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a9fc0-5228-4d90-a1ae-686178b690f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:29:28.202220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a9fc0-5228-4d90-a1ae-686178b690f9 service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:29:28.211776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1277/2549] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:29:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:29:34.578484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f592c-7059-4288-9874-bcbf132b13d0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:34.579071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f592c-7059-4288-9874-bcbf132b13d0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:34.579071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f592c-7059-4288-9874-bcbf132b13d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:34.597706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7a6f592c-7059-4288-9874-bcbf132b13d0 None None] MFA Rules not processed for user `9476865e0b114996b2c0ffce9fca16a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:34.598931 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7a6f592c-7059-4288-9874-bcbf132b13d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:34.634764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:34.658759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1273/2550] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:38.207269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=25 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:44.651696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:46.658076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:46.658617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:46.658617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:46.675233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] MFA Rules not processed for user `059a87a80fc547db8aeb3bb03003cd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:46.676452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:46.683255 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1278/2551] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:46 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 26 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:29:56.670483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:29:56.835228 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d47d-9c7e-4850-9c28-c4d4496a4219 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:29:56.835762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d47d-9c7e-4850-9c28-c4d4496a4219 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:29:56.835880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d77d47d-9c7e-4850-9c28-c4d4496a4219 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:29:56.853174 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d77d47d-9c7e-4850-9c28-c4d4496a4219 None None] MFA Rules not processed for user `92b758b22f054dd991e8af99354a134f`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:29:56.854371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d77d47d-9c7e-4850-9c28-c4d4496a4219 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:29:56.910096 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1274/2552] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:29:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:30:06.903113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=25 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:11.192767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:11.194185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:11.206762 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d21d824-2500-4719-99b4-2fefd0deb7e1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:11.210502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:11.211162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:11.211513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:11.211830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:11.221817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:11.223177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:11.233937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1279/2553] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:30:11 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:30:21.229295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=6 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:30.471822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:30.472307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:30.472475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:30.534092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:30:30.535571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:30.605134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1275/2554] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jul 7 10:30:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:30:30.668099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:30.668099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:30.770504 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e9198a-90f8-46ab-8071-de5c743523b2 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:30.778177 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:30.779300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:30.779882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:30.783677 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:30.794393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:30.796308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:30.807800 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1280/2555] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:30:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:30:38.251939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:38.253584 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:38.365262 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c560608-9287-49c0-acbc-1d2b57837b90 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:38.370323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:38.371336 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:38.371614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:38.372067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] PATH_INFO: `/v3/users/9ae3a6ce6ab4425ebf1180721c10eb2d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:38.373334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae3a6ce6ab4425ebf1180721c10eb2d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:38.375512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:38.426109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae3a6ce6ab4425ebf1180721c10eb2d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.432999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae3a6ce6ab4425ebf1180721c10eb2d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.433695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae3a6ce6ab4425ebf1180721c10eb2d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.441115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae3a6ce6ab4425ebf1180721c10eb2d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.449746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:38.455039 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae3a6ce6ab4425ebf1180721c10eb2d'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.462031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1276/2556] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:30:38 2022] DELETE /identity/v3/users/9ae3a6ce6ab4425ebf1180721c10eb2d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:30:38.631331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:38.642348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92137c8d-b972-4ea0-ba4d-c5ede759c4e4 service swift] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:38.701529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:38.702432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:38.702720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:38.703006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] PATH_INFO: `/v3/projects/5a2481546251413fb65aaed9e3f131e2` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:38.704231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a2481546251413fb65aaed9e3f131e2)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:38.706134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:38.752110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a2481546251413fb65aaed9e3f131e2', 'request_id': 'req-b5ca7e95-4023-411c-908e-f9991144e4c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.759205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a2481546251413fb65aaed9e3f131e2', 'request_id': 'req-b5ca7e95-4023-411c-908e-f9991144e4c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.765369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a2481546251413fb65aaed9e3f131e2', 'request_id': 'req-b5ca7e95-4023-411c-908e-f9991144e4c9'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.772530 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:38.776949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] The token cache is being invalidate because project 5a2481546251413fb65aaed9e3f131e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:30:38.777253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.778335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.780059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1281/2557] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:30:38 2022] DELETE /identity/v3/projects/5a2481546251413fb65aaed9e3f131e2 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:30:38.784095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:38.785493 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:38.799290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc735ae-0c27-40df-9c72-50bc1e40fe61 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:38.857616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:38.858581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:38.858872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:38.859147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] PATH_INFO: `/v3/users/6b280049322e4c51aa1fb890d496d700` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:38.860356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b280049322e4c51aa1fb890d496d700)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:38.862302 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:38.908163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b280049322e4c51aa1fb890d496d700'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.917791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b280049322e4c51aa1fb890d496d700'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.918524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b280049322e4c51aa1fb890d496d700'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.926557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b280049322e4c51aa1fb890d496d700'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.938880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b280049322e4c51aa1fb890d496d700'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:38.946159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1277/2558] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:30:38 2022] DELETE /identity/v3/users/6b280049322e4c51aa1fb890d496d700 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:30:39.120756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:39.122626 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:39.136877 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca7e95-4023-411c-908e-f9991144e4c9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:39.196981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:39.197848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:39.198131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:39.198408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] PATH_INFO: `/v3/projects/deee1a73e228459ea17b72dafe9fd738` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:39.199584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] RBAC: Authorizing `identity:delete_project(project_id=deee1a73e228459ea17b72dafe9fd738)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:39.201508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:39.249622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'deee1a73e228459ea17b72dafe9fd738', 'request_id': 'req-872d7b1f-4928-4ecc-80f2-66cc6daf961b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.257751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'deee1a73e228459ea17b72dafe9fd738', 'request_id': 'req-872d7b1f-4928-4ecc-80f2-66cc6daf961b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.267061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'deee1a73e228459ea17b72dafe9fd738', 'request_id': 'req-872d7b1f-4928-4ecc-80f2-66cc6daf961b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.278787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] The token cache is being invalidate because project deee1a73e228459ea17b72dafe9fd738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:30:39.279079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.279730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.281324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1282/2559] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:30:39 2022] DELETE /identity/v3/projects/deee1a73e228459ea17b72dafe9fd738 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:30:39.721010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:39.722877 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:39.736257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e698bd7e-f689-436d-b421-f276f097b6e7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:39.792265 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:39.793249 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:39.793538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:39.793815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] PATH_INFO: `/v3/users/9476865e0b114996b2c0ffce9fca16a2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:39.794993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=9476865e0b114996b2c0ffce9fca16a2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:39.796949 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:39.809549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:39.842195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9476865e0b114996b2c0ffce9fca16a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.848843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9476865e0b114996b2c0ffce9fca16a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.849480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9476865e0b114996b2c0ffce9fca16a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.856719 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9476865e0b114996b2c0ffce9fca16a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.868685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-728a0055-746b-4914-b331-0a70a4aa05dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9476865e0b114996b2c0ffce9fca16a2'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:39.874443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1278/2560] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:30:39 2022] DELETE /identity/v3/users/9476865e0b114996b2c0ffce9fca16a2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:30:40.054009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.055536 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:40.070595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872d7b1f-4928-4ecc-80f2-66cc6daf961b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.138844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.139649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.140042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.140336 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] PATH_INFO: `/v3/projects/1bd64979a2bc4ca6be57262f4387920e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.141510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bd64979a2bc4ca6be57262f4387920e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.143581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.180511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:40.197457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bd64979a2bc4ca6be57262f4387920e', 'request_id': 'req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:40.204901 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bd64979a2bc4ca6be57262f4387920e', 'request_id': 'req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:40.211975 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bd64979a2bc4ca6be57262f4387920e', 'request_id': 'req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:40.222198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] The token cache is being invalidate because project 1bd64979a2bc4ca6be57262f4387920e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:30:40.222493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:40.223134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-c3ddbe21-18e6-4b3b-b6ba-906d46dc54ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:40.224749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1283/2561] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:30:40 2022] DELETE /identity/v3/projects/1bd64979a2bc4ca6be57262f4387920e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:30:40.230296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240dde-4054-453d-bb4f-ca5909191211 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.230590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240dde-4054-453d-bb4f-ca5909191211 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.230870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26240dde-4054-453d-bb4f-ca5909191211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.249775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-26240dde-4054-453d-bb4f-ca5909191211 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:30:40.250976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-26240dde-4054-453d-bb4f-ca5909191211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:40.309443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1279/2562] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:30:40.317317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c461c5-cd36-4c1f-98be-bcee11fc046e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.317643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c461c5-cd36-4c1f-98be-bcee11fc046e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.317932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c461c5-cd36-4c1f-98be-bcee11fc046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.337391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-42c461c5-cd36-4c1f-98be-bcee11fc046e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:30:40.338610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-42c461c5-cd36-4c1f-98be-bcee11fc046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:40.346617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1284/2563] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:30:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:30:40.350841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240dde-4054-453d-bb4f-ca5909191211 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.360468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26240dde-4054-453d-bb4f-ca5909191211 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.364972 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.365620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.365902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.366177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.366893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:30:40.367201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.368828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.380169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1280/2564] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:30:40.384857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c461c5-cd36-4c1f-98be-bcee11fc046e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.394166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c461c5-cd36-4c1f-98be-bcee11fc046e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.399345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.400042 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.400333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.400613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.401312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.403205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.428388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1285/2565] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:30:40 2022] POST /identity/v3/projects => generated 391 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:30:40.432993 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.442794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed7fc80-940b-409c-9d70-d71eeec3ee7a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.497303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.497969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.498251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.498528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.499208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.501142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.521386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1281/2566] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:30:40 2022] POST /identity/v3/users => generated 296 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:30:40.525490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.535768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188403d-811a-48e9-8aaf-d238c41bf5c7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.540808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.541461 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.541747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.542026 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.542746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.544368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.556206 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1286/2567] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:30:40.559951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.570059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df29d7b-271a-4c7d-8f21-c5b89e1794d7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.574647 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae0448b-619d-4908-82cb-2824bb4edacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.575296 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0448b-619d-4908-82cb-2824bb4edacc admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.575579 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0448b-619d-4908-82cb-2824bb4edacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.575882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae0448b-619d-4908-82cb-2824bb4edacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.576597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0448b-619d-4908-82cb-2824bb4edacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.578179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae0448b-619d-4908-82cb-2824bb4edacc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.579397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:40.590455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1282/2568] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:30:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 07 10:30:40.594247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.604550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa0cb3c-06e0-4f9d-8d76-030d4e75b675 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.609189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.610061 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.610342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.610616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] PATH_INFO: `/v3/projects/35939867f6a44071a74b101fd4d65d62/users/b06d4cb94a924ff98769bd7d3a6b4384/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.625293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35939867f6a44071a74b101fd4d65d62, user_id=b06d4cb94a924ff98769bd7d3a6b4384, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.627935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.640370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1287/2569] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:30:40 2022] PUT /identity/v3/projects/35939867f6a44071a74b101fd4d65d62/users/b06d4cb94a924ff98769bd7d3a6b4384/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:30:40.650515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.650746 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.651024 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.676843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] MFA Rules not processed for user `b06d4cb94a924ff98769bd7d3a6b4384`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:30:40.678072 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:40.741007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1283/2570] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:30:40 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:30:40.751508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:40.753218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:40.809507 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b9bf9-b976-4ab5-9514-41c9e48d3b15 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:40.813190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:40.813838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:40.814126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:40.814405 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:40.823001 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:40.824457 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:40.833251 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1288/2571] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:30:40 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:30:48.992318 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:48.993797 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:49.049958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c41549-e0d2-4799-bcb0-9b7cc7f87338 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:49.053430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fab598-e490-45ef-bbf5-2825f964cf07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:49.054066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab598-e490-45ef-bbf5-2825f964cf07 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:49.054344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab598-e490-45ef-bbf5-2825f964cf07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:49.054616 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fab598-e490-45ef-bbf5-2825f964cf07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:49.056386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-59fab598-e490-45ef-bbf5-2825f964cf07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:49.126444 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fab598-e490-45ef-bbf5-2825f964cf07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:49.127962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fab598-e490-45ef-bbf5-2825f964cf07 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:49.139058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1284/2572] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:30:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:30:50.307060 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:50.308458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:50.320327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ff84e-c9e3-49b2-8039-438fb2c3543d service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:50.323834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:50.324466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:50.324747 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:50.325019 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:50.326675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:50.340086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:50.341490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:50.352746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1289/2573] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:30:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:30:57.143618 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87175ee6-f46c-4d33-9d7b-3f99acf7fcf7 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:57.144287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87175ee6-f46c-4d33-9d7b-3f99acf7fcf7 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:57.144287 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87175ee6-f46c-4d33-9d7b-3f99acf7fcf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:57.161225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-87175ee6-f46c-4d33-9d7b-3f99acf7fcf7 None None] MFA Rules not processed for user `247787cd745444f38953a64780d6ac9b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:30:57.162930 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-87175ee6-f46c-4d33-9d7b-3f99acf7fcf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:57.183410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:57.217805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1285/2574] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:30:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:30:58.764511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:58.773751 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede22fe-e293-435e-9d1a-debab59b6c6d service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:58.777408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb166a85-a766-42d5-9dd5-76aa4a14bee9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:58.778050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb166a85-a766-42d5-9dd5-76aa4a14bee9 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:58.778341 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb166a85-a766-42d5-9dd5-76aa4a14bee9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:58.778617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb166a85-a766-42d5-9dd5-76aa4a14bee9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:58.780404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-fb166a85-a766-42d5-9dd5-76aa4a14bee9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:58.792185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb166a85-a766-42d5-9dd5-76aa4a14bee9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:58.793612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb166a85-a766-42d5-9dd5-76aa4a14bee9 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:58.803176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1290/2575] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:30:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:30:59.078328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87175ee6-f46c-4d33-9d7b-3f99acf7fcf7 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:59.140868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87175ee6-f46c-4d33-9d7b-3f99acf7fcf7 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:59.145497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:59.146376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:59.146718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:59.147500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] PATH_INFO: `/v3/users/b06d4cb94a924ff98769bd7d3a6b4384` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:59.148738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b06d4cb94a924ff98769bd7d3a6b4384)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:59.150667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:59.196808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b06d4cb94a924ff98769bd7d3a6b4384'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.204208 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b06d4cb94a924ff98769bd7d3a6b4384'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.204903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b06d4cb94a924ff98769bd7d3a6b4384'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.212480 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b06d4cb94a924ff98769bd7d3a6b4384'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.225395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b06d4cb94a924ff98769bd7d3a6b4384'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.231736 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1286/2576] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:30:59 2022] DELETE /identity/v3/users/b06d4cb94a924ff98769bd7d3a6b4384 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:30:59.236313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:59.236675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:59.237024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:59.255041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:30:59.256787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:59.287623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:59.321041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1291/2577] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:30:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:30:59.330810 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:59.332399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:59.391824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f116dc-a249-402f-a9e2-c5010020abc8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:59.395629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:59.396309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:59.396595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:59.397410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:59.403464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:30:59.408219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:59.410517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:59.421617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1287/2578] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:30:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:30:59.810008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:30:59.812146 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:30:59.827045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b74e92-c7ee-4421-bc68-5ed949448760 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:30:59.831976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:30:59.832849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:30:59.833135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:30:59.833439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] PATH_INFO: `/v3/projects/35939867f6a44071a74b101fd4d65d62` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:30:59.835258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=35939867f6a44071a74b101fd4d65d62)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:30:59.837200 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:30:59.889293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35939867f6a44071a74b101fd4d65d62', 'request_id': 'req-01b56dc6-513e-4fff-b8f9-f16da1217a4c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.898217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35939867f6a44071a74b101fd4d65d62', 'request_id': 'req-01b56dc6-513e-4fff-b8f9-f16da1217a4c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.904913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35939867f6a44071a74b101fd4d65d62', 'request_id': 'req-01b56dc6-513e-4fff-b8f9-f16da1217a4c'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.915192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] The token cache is being invalidate because project 35939867f6a44071a74b101fd4d65d62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:30:59.915485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.916166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:30:59.917723 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1292/2579] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:30:59 2022] DELETE /identity/v3/projects/35939867f6a44071a74b101fd4d65d62 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:31:04.990818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:04.992329 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:05.049047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99db6d3-9ded-4a69-a876-5752166798b4 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:05.095910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:05.096546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:05.096827 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:05.097168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:05.098849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:05.156022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:05.157662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:05.166821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1288/2580] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:04 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:09.912126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:15.162701 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=71 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:23.438768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:23.440554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:23.523340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b56dc6-513e-4fff-b8f9-f16da1217a4c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:23.529404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:23.530101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:23.530390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:23.530884 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:23.532896 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:23.546468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:23.548014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:23.559773 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1293/2581] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:24.213164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:24.214614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:24.269603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbddc6b-5b89-4f06-85dc-76cf1709f8a2 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:24.273194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:24.273828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:24.274109 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:24.274387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:24.276129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:24.338134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:24.339502 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:24.349261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1289/2582] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:27.310348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:27.312008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:27.372981 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:27.374665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:27.393558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2517ab95-d112-41b6-8b88-85096ebbae9b service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:27.398134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:27.399035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:27.399311 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:27.399674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] PATH_INFO: `/v3/users/92b758b22f054dd991e8af99354a134f` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:27.400903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92b758b22f054dd991e8af99354a134f)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:27.402820 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:27.403035 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfb3c7-8d61-447a-9f5f-ac6f1c76295d service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:27.408737 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:27.409388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:27.409676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:27.409959 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:27.411756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:27.424315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:27.425908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:27.436812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1294/2583] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:27.465436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92b758b22f054dd991e8af99354a134f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.472672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92b758b22f054dd991e8af99354a134f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.473331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92b758b22f054dd991e8af99354a134f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.480363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92b758b22f054dd991e8af99354a134f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.492654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99850922-1c78-441c-b43f-87bf8f6fc936 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92b758b22f054dd991e8af99354a134f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.498476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1290/2584] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:31:27 2022] DELETE /identity/v3/users/92b758b22f054dd991e8af99354a134f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:31:27.668007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:27.679519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c73e27-7fb3-4f9f-b5c4-0fb228067bde service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:27.735508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:27.736333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:27.736618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:27.736894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] PATH_INFO: `/v3/projects/0a883b49db0f4f32a32bf6d87e2f6104` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:27.738052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a883b49db0f4f32a32bf6d87e2f6104)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:27.739983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:27.756512 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=100 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:27.784615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a883b49db0f4f32a32bf6d87e2f6104', 'request_id': 'req-d63ff45d-435d-403b-b0a1-4b95aa14ebde'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.792943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a883b49db0f4f32a32bf6d87e2f6104', 'request_id': 'req-d63ff45d-435d-403b-b0a1-4b95aa14ebde'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.798984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a883b49db0f4f32a32bf6d87e2f6104', 'request_id': 'req-d63ff45d-435d-403b-b0a1-4b95aa14ebde'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.809692 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] The token cache is being invalidate because project 0a883b49db0f4f32a32bf6d87e2f6104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:31:27.809974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.810624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d63ff45d-435d-403b-b0a1-4b95aa14ebde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:31:27.812458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1295/2585] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:31:27 2022] DELETE /identity/v3/projects/0a883b49db0f4f32a32bf6d87e2f6104 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:31:27.817163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0b1e-4f7e-4631-8674-a918a3981f2c None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:27.817627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0b1e-4f7e-4631-8674-a918a3981f2c None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:27.817816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c0b1e-4f7e-4631-8674-a918a3981f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:27.835146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c73c0b1e-4f7e-4631-8674-a918a3981f2c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:31:27.836398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c73c0b1e-4f7e-4631-8674-a918a3981f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:27.853140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:27.896100 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1291/2586] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:31:27.904238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154244dc-975c-418c-8b6f-57f1b1666413 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:27.904534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154244dc-975c-418c-8b6f-57f1b1666413 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:27.904815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154244dc-975c-418c-8b6f-57f1b1666413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:27.923064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-154244dc-975c-418c-8b6f-57f1b1666413 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:31:27.924287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-154244dc-975c-418c-8b6f-57f1b1666413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:27.932084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1296/2587] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:31:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:31:27.936322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0b1e-4f7e-4631-8674-a918a3981f2c None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:27.946255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c0b1e-4f7e-4631-8674-a918a3981f2c None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:27.950889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:27.951564 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:27.951881 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:27.952156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:27.952869 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:31:27.953173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:27.954761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:27.966655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1292/2588] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:31:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:31:27.970758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154244dc-975c-418c-8b6f-57f1b1666413 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:27.980487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154244dc-975c-418c-8b6f-57f1b1666413 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:27.985086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:27.985789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:27.986012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:27.986290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:27.986980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:27.988904 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:28.013908 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1297/2589] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:31:27 2022] POST /identity/v3/projects => generated 389 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:31:28.017862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:28.028674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40bbcb-93f0-4496-8b70-2726eac1e234 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:28.087423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:28.088142 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:28.088422 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:28.088694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:28.089383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:28.091294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:28.112012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1293/2590] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:31:28 2022] POST /identity/v3/users => generated 295 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:31:28.116220 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:28.126579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2db17d8-7940-46b5-b1cb-60bf51938e67 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:28.131287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:28.132099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:28.132472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:28.132759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:28.133490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:28.135086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:28.145887 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1298/2591] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:31:28.150529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:28.161743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9800bf45-0691-47a5-b66d-b68905696214 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:28.167323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17354c2-d1b0-46e3-a1bc-6b249dd0a07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:28.168084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17354c2-d1b0-46e3-a1bc-6b249dd0a07e admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:28.168371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17354c2-d1b0-46e3-a1bc-6b249dd0a07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:28.168649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17354c2-d1b0-46e3-a1bc-6b249dd0a07e admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:28.169368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17354c2-d1b0-46e3-a1bc-6b249dd0a07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:28.170957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17354c2-d1b0-46e3-a1bc-6b249dd0a07e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:28.180925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1294/2592] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:31:28.185202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:28.195557 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a4960-3688-40e2-bebb-c1b53b056727 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:28.200150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:28.201024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:28.201307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:28.201581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] PATH_INFO: `/v3/projects/96923df6a03841498e3732c8504b6363/users/31bcb5c27d8b4ae2b9a3207fff0eabfe/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:28.215974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96923df6a03841498e3732c8504b6363, user_id=31bcb5c27d8b4ae2b9a3207fff0eabfe, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:28.218597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:28.231608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1299/2593] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:31:28 2022] PUT /identity/v3/projects/96923df6a03841498e3732c8504b6363/users/31bcb5c27d8b4ae2b9a3207fff0eabfe/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:31:28.236555 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132faa74-be4b-4749-a025-678896a37192 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:28.236814 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132faa74-be4b-4749-a025-678896a37192 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:28.237093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132faa74-be4b-4749-a025-678896a37192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:28.254308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-132faa74-be4b-4749-a025-678896a37192 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:31:28.255507 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-132faa74-be4b-4749-a025-678896a37192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:28.312986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1295/2594] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:31:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:31:28.323361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:28.325140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:28.380845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69718c95-412b-4332-84c3-27ca2ebf75f8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:28.385014 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:28.385653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:28.385937 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:28.386213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:28.394954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:28.396384 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:28.406277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1300/2595] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:32.273331 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1c13f-fc2f-4d4e-be85-15555707f423 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:32.273809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1c13f-fc2f-4d4e-be85-15555707f423 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:32.273898 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1c13f-fc2f-4d4e-be85-15555707f423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:32.306127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-bbd1c13f-fc2f-4d4e-be85-15555707f423 None None] MFA Rules not processed for user `31bcb5c27d8b4ae2b9a3207fff0eabfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:31:32.307825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-bbd1c13f-fc2f-4d4e-be85-15555707f423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:32.327199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:32.382481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1296/2596] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:31:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:31:32.396979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:32.399109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:32.452472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:32.482350 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c910b694-826b-4d8f-afb6-ef125950486e service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:32.488215 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:32.488886 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:32.489173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:32.489452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:32.498172 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:32.499806 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:32.509456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1301/2597] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:33.768900 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:33.769462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:33.769462 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:33.789774 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] MFA Rules not processed for user `31bcb5c27d8b4ae2b9a3207fff0eabfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:31:33.791000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:33.798131 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1297/2598] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:31:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:31:34.123493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:34.125085 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:34.137675 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31e6748-a2b5-42e6-8c26-9b7fa4b290bd service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:34.141898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:34.142586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:34.143321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:34.143610 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:34.145843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:34.156483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:34.157984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:34.166863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1302/2599] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:34.566130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:34.568032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:34.680668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8359c9ef-1faf-48c0-a7ba-1910fa9ce2f4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:34.686642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:34.687315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:34.687600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:34.687944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:34.689925 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:34.703887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:34.705512 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:34.717896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1298/2600] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:36.441425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:36.451501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a3f0e8-f50b-4d3a-a342-77ba8a340f91 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:36.455189 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:36.456052 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:36.456340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:36.456620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:36.458448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:36.469068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:36.470519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:36.479282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1303/2601] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:42.273992 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:42.286961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89360c70-6389-45c2-a650-e1c48d7d3054 service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:42.293576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19d393-eb0b-48d0-a809-610834875c8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:42.294230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19d393-eb0b-48d0-a809-610834875c8c service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:42.294633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19d393-eb0b-48d0-a809-610834875c8c service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:42.294917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19d393-eb0b-48d0-a809-610834875c8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:42.296728 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-8d19d393-eb0b-48d0-a809-610834875c8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:42.309818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19d393-eb0b-48d0-a809-610834875c8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:42.311409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19d393-eb0b-48d0-a809-610834875c8c service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:42.324195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1299/2602] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:42 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:31:45.275263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:45.277541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:45.296506 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b849101-6c60-4e9f-bfa7-b2234b72a203 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:45.302322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e107f3a7-e0a7-446f-b9db-53458918f6ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:45.303594 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107f3a7-e0a7-446f-b9db-53458918f6ba service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:45.304015 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107f3a7-e0a7-446f-b9db-53458918f6ba service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:45.304399 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e107f3a7-e0a7-446f-b9db-53458918f6ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:45.306383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e107f3a7-e0a7-446f-b9db-53458918f6ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:45.320275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107f3a7-e0a7-446f-b9db-53458918f6ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:45.321957 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e107f3a7-e0a7-446f-b9db-53458918f6ba service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:45.332964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1304/2603] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:52.319678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=60 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:55.328946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:31:58.547815 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a952a96-6c37-4924-9ed8-e67c66f90b43 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:58.548399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a952a96-6c37-4924-9ed8-e67c66f90b43 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:58.548399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a952a96-6c37-4924-9ed8-e67c66f90b43 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:58.549742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1300/2604] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:31:58.556703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a639e60-00f2-4879-a50b-6b3fa551a389 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:58.557190 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a639e60-00f2-4879-a50b-6b3fa551a389 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:58.557300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a639e60-00f2-4879-a50b-6b3fa551a389 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:58.577400 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4a639e60-00f2-4879-a50b-6b3fa551a389 None None] MFA Rules not processed for user `61c23b3898c14a99bd1521644345d6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:31:58.578624 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4a639e60-00f2-4879-a50b-6b3fa551a389 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:58.585526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1305/2605] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:31:58 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:31:58.597846 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a952a96-6c37-4924-9ed8-e67c66f90b43 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:58.610057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a952a96-6c37-4924-9ed8-e67c66f90b43 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:58.613720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369604a5-9d42-441e-aefe-0e2de29982ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:58.614365 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369604a5-9d42-441e-aefe-0e2de29982ee service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:58.614643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369604a5-9d42-441e-aefe-0e2de29982ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:58.614936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369604a5-9d42-441e-aefe-0e2de29982ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:58.623317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369604a5-9d42-441e-aefe-0e2de29982ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:58.625007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369604a5-9d42-441e-aefe-0e2de29982ee service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:58.632624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1301/2606] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:31:58.713503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fefe0c-ed7d-4eb5-87f7-d1ca4f7669b9 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:58.713804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fefe0c-ed7d-4eb5-87f7-d1ca4f7669b9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:58.714087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fefe0c-ed7d-4eb5-87f7-d1ca4f7669b9 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:58.715476 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1306/2607] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:31:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:31:58.721487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f594eb-2e87-4494-b552-9587c26436e4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:58.721780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f594eb-2e87-4494-b552-9587c26436e4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:58.722115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f594eb-2e87-4494-b552-9587c26436e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:58.758915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c8f594eb-2e87-4494-b552-9587c26436e4 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:31:58.760162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c8f594eb-2e87-4494-b552-9587c26436e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:31:58.769484 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1302/2608] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:31:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:31:58.780355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fefe0c-ed7d-4eb5-87f7-d1ca4f7669b9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:31:58.791977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fefe0c-ed7d-4eb5-87f7-d1ca4f7669b9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:31:58.795604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:31:58.796276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:31:58.796567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:31:58.796854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:31:58.807110 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:31:58.808559 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:31:58.819571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1307/2609] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:31:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:32:08.756509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=15 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:08.814734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=9 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:20.275504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f594eb-2e87-4494-b552-9587c26436e4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:20.285013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f594eb-2e87-4494-b552-9587c26436e4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:20.288842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:20.289541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:20.289823 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:20.290095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:20.291910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:20.303595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:20.305686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:20.315709 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1303/2610] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:32:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:32:23.371570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:23.373769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:23.393646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0702b4b4-2cea-4d44-9e62-3b70c765c831 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:23.398246 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:23.399724 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:23.400041 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:23.400318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] PATH_INFO: `/v3/users/247787cd745444f38953a64780d6ac9b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:23.401498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] RBAC: Authorizing `identity:delete_user(user_id=247787cd745444f38953a64780d6ac9b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:23.403404 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:23.451570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '247787cd745444f38953a64780d6ac9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.458907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '247787cd745444f38953a64780d6ac9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.459759 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '247787cd745444f38953a64780d6ac9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.472191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '247787cd745444f38953a64780d6ac9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.479057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-2aef466d-1b5a-454b-aed6-acbb7dfae588 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '247787cd745444f38953a64780d6ac9b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.484942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1308/2611] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:32:23 2022] DELETE /identity/v3/users/247787cd745444f38953a64780d6ac9b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:32:23.655882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:23.667872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d296a118-06a1-4e4d-aea0-3ee911db30e7 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:23.725176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:23.725969 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:23.726251 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:23.727171 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] PATH_INFO: `/v3/projects/98924fde961c41358ffb24ac3819557a` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:23.728380 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98924fde961c41358ffb24ac3819557a)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:23.730324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:23.774371 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98924fde961c41358ffb24ac3819557a', 'request_id': 'req-b985555f-2c52-4e90-bfdf-be0345e12e06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.780628 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98924fde961c41358ffb24ac3819557a', 'request_id': 'req-b985555f-2c52-4e90-bfdf-be0345e12e06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.787923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98924fde961c41358ffb24ac3819557a', 'request_id': 'req-b985555f-2c52-4e90-bfdf-be0345e12e06'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.798482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] The token cache is being invalidate because project 98924fde961c41358ffb24ac3819557a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:32:23.798772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.799399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-b985555f-2c52-4e90-bfdf-be0345e12e06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:32:23.800923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1304/2612] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:32:23 2022] DELETE /identity/v3/projects/98924fde961c41358ffb24ac3819557a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:32:23.805843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a8b2-08d2-4c9e-af9e-06ad7db0e500 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:23.806132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a8b2-08d2-4c9e-af9e-06ad7db0e500 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:23.806411 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0339a8b2-08d2-4c9e-af9e-06ad7db0e500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:23.823705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-0339a8b2-08d2-4c9e-af9e-06ad7db0e500 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:23.824924 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0339a8b2-08d2-4c9e-af9e-06ad7db0e500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:23.882701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1309/2613] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:23.890951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb84545-7f1c-4977-86b7-87e908c65aba None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:23.891241 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb84545-7f1c-4977-86b7-87e908c65aba None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:23.891520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb84545-7f1c-4977-86b7-87e908c65aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:23.910351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-edb84545-7f1c-4977-86b7-87e908c65aba None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:23.911558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-edb84545-7f1c-4977-86b7-87e908c65aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:23.920082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1305/2614] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:32:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:23.924093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a8b2-08d2-4c9e-af9e-06ad7db0e500 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:23.933697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0339a8b2-08d2-4c9e-af9e-06ad7db0e500 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:23.938295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:23.938947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:23.939237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:23.939546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:23.940316 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:32:23.940630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:23.942214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:23.953812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1310/2615] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:32:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:32:23.958326 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb84545-7f1c-4977-86b7-87e908c65aba None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:23.967953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb84545-7f1c-4977-86b7-87e908c65aba None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:23.972532 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:23.973190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:23.973470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:23.973742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:23.974424 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:23.976382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:24.011264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1306/2616] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:32:23 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:32:24.015280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:24.025571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3720d1-6781-4ccd-be6c-392bab04af5f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:24.083644 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:24.084351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:24.084636 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:24.084913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:24.085601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:24.088159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:24.094757 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:24.110657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1311/2617] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:32:24 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:32:24.114435 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:24.125041 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403fab1-0502-499c-acac-09a08ba30f3e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:24.129505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:24.130192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:24.130476 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:24.130752 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:24.131471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:24.133625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:24.144460 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1307/2618] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:32:24.149422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:24.159517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e92a9-7484-4646-be78-5f3426ccba6c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:24.164293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7631f6b6-51e0-457a-9929-1fe55e221327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:24.164986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631f6b6-51e0-457a-9929-1fe55e221327 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:24.165280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631f6b6-51e0-457a-9929-1fe55e221327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:24.165577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7631f6b6-51e0-457a-9929-1fe55e221327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:24.166550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631f6b6-51e0-457a-9929-1fe55e221327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:24.168643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7631f6b6-51e0-457a-9929-1fe55e221327 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:24.183618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1312/2619] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:32:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:32:24.184901 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:24.195767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c0bab1-3eff-4970-b398-4b12ae044c49 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:24.200394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:24.201268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:24.201545 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:24.201816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] PATH_INFO: `/v3/projects/1568d5b1c4bb471f9de53f4d5cf7cc82/users/1222d8bb50554d64ac674f109b6ad077/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:24.216653 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1568d5b1c4bb471f9de53f4d5cf7cc82, user_id=1222d8bb50554d64ac674f109b6ad077, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:24.219292 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:24.231779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1308/2620] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:32:24 2022] PUT /identity/v3/projects/1568d5b1c4bb471f9de53f4d5cf7cc82/users/1222d8bb50554d64ac674f109b6ad077/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:32:24.236710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873645-1dbe-4235-94e0-d8cd918aa09d None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:24.237006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873645-1dbe-4235-94e0-d8cd918aa09d None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:24.237291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f873645-1dbe-4235-94e0-d8cd918aa09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:24.254740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-8f873645-1dbe-4235-94e0-d8cd918aa09d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:24.255965 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8f873645-1dbe-4235-94e0-d8cd918aa09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:24.311926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1313/2621] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:32:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:24.322189 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:24.323864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:24.330166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:24.379947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6c82f-8e16-4222-8037-ffd1ed328f3a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:24.384255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:24.384893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:24.385170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:24.385441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:24.394297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:24.395718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:24.405445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1309/2622] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:32:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:32:28.373807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0801-b8c0-4998-ab81-af26ee1271a4 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:28.374301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0801-b8c0-4998-ab81-af26ee1271a4 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:28.374395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc0801-b8c0-4998-ab81-af26ee1271a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:28.402608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-06cc0801-b8c0-4998-ab81-af26ee1271a4 None None] MFA Rules not processed for user `1222d8bb50554d64ac674f109b6ad077`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:28.403964 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-06cc0801-b8c0-4998-ab81-af26ee1271a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:28.486257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1314/2623] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:32:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:28.494961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:28.496986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:28.578376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afceb64-602c-4866-9cba-1c45e51878ef service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:28.584600 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:28.585270 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:28.585603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:28.585865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:28.594660 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:28.596288 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:28.607133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1310/2624] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:32:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:32:29.966375 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0164f-aff4-421f-8d0f-af4a8da5e0bf None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:29.966826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0164f-aff4-421f-8d0f-af4a8da5e0bf None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:29.966947 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0164f-aff4-421f-8d0f-af4a8da5e0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:29.988057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-61a0164f-aff4-421f-8d0f-af4a8da5e0bf None None] MFA Rules not processed for user `1222d8bb50554d64ac674f109b6ad077`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:29.989569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-61a0164f-aff4-421f-8d0f-af4a8da5e0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:29.999025 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1315/2625] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:32:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:30.549720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:30.564278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350e28a-d5dc-4c49-ab46-e4b0cf90ba2c service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:30.569421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b927e3-2dbd-4508-af3e-2fb21c879884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:30.570740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b927e3-2dbd-4508-af3e-2fb21c879884 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:30.571122 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b927e3-2dbd-4508-af3e-2fb21c879884 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:30.571497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b927e3-2dbd-4508-af3e-2fb21c879884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:30.574178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6b927e3-2dbd-4508-af3e-2fb21c879884 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:30.589530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b927e3-2dbd-4508-af3e-2fb21c879884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:30.591514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b927e3-2dbd-4508-af3e-2fb21c879884 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:30.606646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1311/2626] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:32:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:32:32.407030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:32.407514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:32.408380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:32.459032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] MFA Rules not processed for user `059a87a80fc547db8aeb3bb03003cd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:32.460430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:32.541606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1316/2627] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:32:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:40.600601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=64 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:42.534108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=3,keystone:SELECT=94 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:44.958463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d0388-2e15-431b-84a0-a441bd73e97e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:44.959508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d0388-2e15-431b-84a0-a441bd73e97e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:44.960013 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec1d0388-2e15-431b-84a0-a441bd73e97e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:44.982431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ec1d0388-2e15-431b-84a0-a441bd73e97e None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:44.984003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ec1d0388-2e15-431b-84a0-a441bd73e97e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:45.049116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1312/2628] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:32:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:45.101322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:45.103344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:45.189918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f386a21-7d90-45eb-9efc-70878b470566 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:45.195395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7f84d-2033-4c7b-8adb-40475c2abed8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:45.196091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f84d-2033-4c7b-8adb-40475c2abed8 service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:45.196582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f84d-2033-4c7b-8adb-40475c2abed8 service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:45.196862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7f84d-2033-4c7b-8adb-40475c2abed8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:45.205561 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7f84d-2033-4c7b-8adb-40475c2abed8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:45.207157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7f84d-2033-4c7b-8adb-40475c2abed8 service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:45.216605 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1317/2629] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:32:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:32:55.040429 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=25 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:55.212993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:32:58.825094 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d14f9-8656-4db1-9bf7-3e63f448c062 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:58.825657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d14f9-8656-4db1-9bf7-3e63f448c062 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:58.825861 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d14f9-8656-4db1-9bf7-3e63f448c062 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:58.827348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1313/2630] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:32:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:32:58.833728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3af60-7c8c-4be1-aefc-30bd36af1400 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:58.834187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3af60-7c8c-4be1-aefc-30bd36af1400 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:58.834187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3af60-7c8c-4be1-aefc-30bd36af1400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:58.853432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-a6d3af60-7c8c-4be1-aefc-30bd36af1400 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:58.854664 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-a6d3af60-7c8c-4be1-aefc-30bd36af1400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:58.864043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1318/2631] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:32:58 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:32:58.876522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d14f9-8656-4db1-9bf7-3e63f448c062 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:58.878317 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a77d14f9-8656-4db1-9bf7-3e63f448c062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:58.964896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d14f9-8656-4db1-9bf7-3e63f448c062 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:58.968437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784d5b52-87a4-4ab3-a3a5-7fad499f05c8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:58.969079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d5b52-87a4-4ab3-a3a5-7fad499f05c8 service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:58.969362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d5b52-87a4-4ab3-a3a5-7fad499f05c8 service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:58.969692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784d5b52-87a4-4ab3-a3a5-7fad499f05c8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:58.980337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d5b52-87a4-4ab3-a3a5-7fad499f05c8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:58.981757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784d5b52-87a4-4ab3-a3a5-7fad499f05c8 service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:58.992965 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1314/2632] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:32:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:32:59.056802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf59105-3f79-46b0-b03f-47a4e597ca76 None None] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:59.057116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf59105-3f79-46b0-b03f-47a4e597ca76 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:59.057398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf59105-3f79-46b0-b03f-47a4e597ca76 None None] PATH_INFO: `/` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:59.058833 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1319/2633] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:32:59.063481 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b993-0c8e-480f-a2d8-cc352aee5465 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:59.063800 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b993-0c8e-480f-a2d8-cc352aee5465 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:59.064171 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef13b993-0c8e-480f-a2d8-cc352aee5465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:59.084179 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-ef13b993-0c8e-480f-a2d8-cc352aee5465 None None] MFA Rules not processed for user `61c23b3898c14a99bd1521644345d6ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:59.085529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ef13b993-0c8e-480f-a2d8-cc352aee5465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:59.092642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1315/2634] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:32:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:59.104252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf59105-3f79-46b0-b03f-47a4e597ca76 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:59.115185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf59105-3f79-46b0-b03f-47a4e597ca76 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:59.118686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d275b45-e9f3-4b42-a6e6-85ac97d5a430 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:59.119326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d275b45-e9f3-4b42-a6e6-85ac97d5a430 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:59.119611 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d275b45-e9f3-4b42-a6e6-85ac97d5a430 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:59.119921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d275b45-e9f3-4b42-a6e6-85ac97d5a430 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:59.126802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d275b45-e9f3-4b42-a6e6-85ac97d5a430 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:59.128268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d275b45-e9f3-4b42-a6e6-85ac97d5a430 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:59.136592 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1320/2635] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:32:59 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:32:59.217232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3097968-490e-4357-b49a-052db83aa759 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:59.217517 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3097968-490e-4357-b49a-052db83aa759 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:59.217841 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3097968-490e-4357-b49a-052db83aa759 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:59.219198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1316/2636] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:32:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 07 10:32:59.224095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbe4cc-0435-4654-85f3-09345ca1fdb5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:59.224390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbe4cc-0435-4654-85f3-09345ca1fdb5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:59.224672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbbe4cc-0435-4654-85f3-09345ca1fdb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:59.242774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9fbbe4cc-0435-4654-85f3-09345ca1fdb5 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:32:59.244011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9fbbe4cc-0435-4654-85f3-09345ca1fdb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:59.253128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1321/2637] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:32:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:32:59.263786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3097968-490e-4357-b49a-052db83aa759 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:32:59.265411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f3097968-490e-4357-b49a-052db83aa759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:32:59.279137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3097968-490e-4357-b49a-052db83aa759 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:32:59.282667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:32:59.283291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:32:59.283569 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:32:59.283876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:32:59.293645 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:32:59.295052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:32:59.305696 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1317/2638] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:32:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:33:01.477519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3047-8356-486d-a9ff-4a3fa355e1b2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:01.477979 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3047-8356-486d-a9ff-4a3fa355e1b2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:01.478072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b3047-8356-486d-a9ff-4a3fa355e1b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:01.495602 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4b2b3047-8356-486d-a9ff-4a3fa355e1b2 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:33:01.496805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4b2b3047-8356-486d-a9ff-4a3fa355e1b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:01.504196 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1322/2639] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:33:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:33:01.515893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:01.527176 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937e9b9-8373-42f2-a76c-9846ca5b6aae service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:01.532112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:01.532748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:01.533031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:01.533304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:01.541862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:01.543379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:01.551467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1318/2640] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:33:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:33:11.489963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=15 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:33:11.548143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=50 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:33:22.612989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd3ed5-07a8-43a4-a26a-dd19f56355e8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:22.613463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd3ed5-07a8-43a4-a26a-dd19f56355e8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:22.613562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcd3ed5-07a8-43a4-a26a-dd19f56355e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:22.630902 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-9bcd3ed5-07a8-43a4-a26a-dd19f56355e8 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:33:22.632151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9bcd3ed5-07a8-43a4-a26a-dd19f56355e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:22.639298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1323/2641] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jul 7 10:33:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:33:22.651825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:22.662531 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be72c2d4-7177-4be1-a75e-7bb9f7ddcad5 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:22.668074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:22.668706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:22.668987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:22.669263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:22.677159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:22.678752 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:22.686663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1319/2642] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:33:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:33:27.154321 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd3ed5-07a8-43a4-a26a-dd19f56355e8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:27.164082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcd3ed5-07a8-43a4-a26a-dd19f56355e8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:27.167607 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:27.168284 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:27.168570 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:27.168849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:27.170612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:27.228055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:27.229486 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:27.238253 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1324/2643] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:33:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:33:31.943791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:31.953247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a1781c-c66c-4562-a510-44c62e067667 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:31.956834 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:31.957468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:31.957750 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:31.958027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:31.960412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:31.972004 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:31.973427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:31.982915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1320/2644] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:33:34.997514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:34.998984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:35.089650 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b608a8-c44d-453e-84e6-eb4cb4babddd service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:35.094634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:35.095294 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:35.095581 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:35.095895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:35.097687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:35.109589 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:35.111244 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:35.121544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1325/2645] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:33:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:33:38.180514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:38.182708 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:38.201047 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a992203a-f236-43da-b931-f346a5ef7a2f service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:38.212141 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:38.212938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:38.216183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:38.216724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] PATH_INFO: `/v3/users/d8c69ecf2f23490c92ab315433273817` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:38.219151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8c69ecf2f23490c92ab315433273817)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:38.221194 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:38.282878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8c69ecf2f23490c92ab315433273817'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.292017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8c69ecf2f23490c92ab315433273817'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.292812 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8c69ecf2f23490c92ab315433273817'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.300761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8c69ecf2f23490c92ab315433273817'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.316589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8c69ecf2f23490c92ab315433273817'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.325755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1321/2646] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:33:38 2022] DELETE /identity/v3/users/d8c69ecf2f23490c92ab315433273817 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:33:38.535514 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:38.549340 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91389897-4a6f-48b9-99fa-a49af785d679 service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:38.611454 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:38.612365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:38.612649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:38.613139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] PATH_INFO: `/v3/projects/21dabce5b3274cfcba5840982b59434e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:38.614344 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21dabce5b3274cfcba5840982b59434e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:38.616850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:38.646071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:33:38.670328 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21dabce5b3274cfcba5840982b59434e', 'request_id': 'req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.677430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21dabce5b3274cfcba5840982b59434e', 'request_id': 'req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.683845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21dabce5b3274cfcba5840982b59434e', 'request_id': 'req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.696898 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] The token cache is being invalidate because project 21dabce5b3274cfcba5840982b59434e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:33:38.697192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.697866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-a78f5dec-a28f-4354-8c16-5e9d9de3d5d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.699443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1326/2647] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:33:38 2022] DELETE /identity/v3/projects/21dabce5b3274cfcba5840982b59434e => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:33:38.703384 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:38.704896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:38.718625 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a64b8e-2ee2-4d41-acf1-cdce772ca70e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:38.778341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:38.779225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:38.779506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:38.779816 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] PATH_INFO: `/v3/users/ac0519a709744125a2dad4d45bcc922b` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:38.781030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac0519a709744125a2dad4d45bcc922b)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:38.782929 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:38.830354 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac0519a709744125a2dad4d45bcc922b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.832170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:33:38.840398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac0519a709744125a2dad4d45bcc922b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.841345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac0519a709744125a2dad4d45bcc922b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.848811 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac0519a709744125a2dad4d45bcc922b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.861070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac0519a709744125a2dad4d45bcc922b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:38.867147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1322/2648] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:33:38 2022] DELETE /identity/v3/users/ac0519a709744125a2dad4d45bcc922b => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:33:38.992657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:38.992955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:38.993293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:39.011248 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] MFA Rules not processed for user `1222d8bb50554d64ac674f109b6ad077`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:33:39.012491 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:39.046016 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:39.047650 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:39.074153 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1327/2649] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:33:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:33:39.128344 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba824c9-3a2e-4222-b4d9-9ef2a476c721 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:39.133670 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:39.134534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:39.134813 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:39.135084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] PATH_INFO: `/v3/projects/7a764964d68c403b954db4aa89a05947` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:39.136530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a764964d68c403b954db4aa89a05947)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:39.138455 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:39.187092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a764964d68c403b954db4aa89a05947', 'request_id': 'req-d327e257-f55d-4518-80c2-e80c590e394e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:39.194295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a764964d68c403b954db4aa89a05947', 'request_id': 'req-d327e257-f55d-4518-80c2-e80c590e394e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:39.203012 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a764964d68c403b954db4aa89a05947', 'request_id': 'req-d327e257-f55d-4518-80c2-e80c590e394e'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:39.214071 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] The token cache is being invalidate because project 7a764964d68c403b954db4aa89a05947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:33:39.214363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:39.215001 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d327e257-f55d-4518-80c2-e80c590e394e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:33:39.216786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1323/2650] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:33:39 2022] DELETE /identity/v3/projects/7a764964d68c403b954db4aa89a05947 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:33:49.066205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=40,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:33:49.210886 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=63 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:33:54.813945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:33:54.815406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:54.882871 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899e13ac-3104-4115-940d-c7f4409fea21 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:33:54.939548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:33:54.940240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:33:54.940529 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:33:54.940805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:33:54.942819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:33:54.954525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:33:54.956131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:33:54.965844 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1328/2651] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:33:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:34:04.961698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=58 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:34:09.767771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526e33-ce40-47d2-ac71-6edbe560f195 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:34:09.768308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526e33-ce40-47d2-ac71-6edbe560f195 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:34:09.768475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68526e33-ce40-47d2-ac71-6edbe560f195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:34:09.786394 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-68526e33-ce40-47d2-ac71-6edbe560f195 None None] MFA Rules not processed for user `31bcb5c27d8b4ae2b9a3207fff0eabfe`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:34:09.787594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-68526e33-ce40-47d2-ac71-6edbe560f195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:34:09.841266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1324/2652] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:34:09 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:34:19.834169 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=25 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:34:30.125861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:34:30.127281 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:34:30.184808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc0201-7ef7-4597-85db-6729fccd0afc service placement] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:34:30.188425 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1dd822-7061-402c-a0af-f874e43acbcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:34:30.189101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd822-7061-402c-a0af-f874e43acbcf service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:34:30.189380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd822-7061-402c-a0af-f874e43acbcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:34:30.189655 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1dd822-7061-402c-a0af-f874e43acbcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:34:30.191766 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5e1dd822-7061-402c-a0af-f874e43acbcf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:34:30.262638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dd822-7061-402c-a0af-f874e43acbcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:34:30.262638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1dd822-7061-402c-a0af-f874e43acbcf service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:34:30.275498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1329/2653] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:34:30 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:34:38.311356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3b6e-d59f-4636-896d-eeab3756ed7f None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:34:38.311935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3b6e-d59f-4636-896d-eeab3756ed7f None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:34:38.312195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e3b6e-d59f-4636-896d-eeab3756ed7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:34:38.329891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-7e7e3b6e-d59f-4636-896d-eeab3756ed7f None None] MFA Rules not processed for user `059a87a80fc547db8aeb3bb03003cd2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:34:38.331099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-7e7e3b6e-d59f-4636-896d-eeab3756ed7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:34:38.338634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1325/2654] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:34:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:34:40.268083 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=48 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:34:40.434919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:34:40.435286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:34:40.435519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:34:40.454084 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] MFA Rules not processed for user `1222d8bb50554d64ac674f109b6ad077`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:34:40.455302 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:34:40.523210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1330/2655] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:34:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:34:48.324556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:34:50.513046 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=25 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:35:26.541596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530fca5-6a3f-4909-9e45-2adc8e90bc12 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:26.542105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530fca5-6a3f-4909-9e45-2adc8e90bc12 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:26.542217 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d530fca5-6a3f-4909-9e45-2adc8e90bc12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:26.560067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d530fca5-6a3f-4909-9e45-2adc8e90bc12 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:35:26.561264 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d530fca5-6a3f-4909-9e45-2adc8e90bc12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:26.615237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1326/2656] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:35:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:35:26.660962 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:35:26.662387 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:26.731442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b67969e-1861-4189-b26a-5f44b8fd0334 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:35:26.736276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669aab6-9006-45f7-aa1d-d7979ea2ccaa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:35:26.736923 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669aab6-9006-45f7-aa1d-d7979ea2ccaa service swift] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:26.737210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669aab6-9006-45f7-aa1d-d7979ea2ccaa service swift] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:26.737533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669aab6-9006-45f7-aa1d-d7979ea2ccaa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:26.746176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669aab6-9006-45f7-aa1d-d7979ea2ccaa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:35:26.747763 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9669aab6-9006-45f7-aa1d-d7979ea2ccaa service swift] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:35:26.755734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1331/2657] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:35:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:35:36.608617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=25 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:35:36.752546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=32 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:35:40.055572 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c3e77-fd0a-4744-bf84-28d404e09dd6 None None] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:40.056139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c3e77-fd0a-4744-bf84-28d404e09dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:40.056139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287c3e77-fd0a-4744-bf84-28d404e09dd6 None None] PATH_INFO: `/` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:40.057533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1327/2658] 10.222.0.16 () {54 vars in 921 bytes} [Thu Jul 7 10:35:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 07 10:35:40.062627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dd083-a510-4f5f-a774-15611cf85315 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:40.063118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dd083-a510-4f5f-a774-15611cf85315 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:40.063226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888dd083-a510-4f5f-a774-15611cf85315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:40.083090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-888dd083-a510-4f5f-a774-15611cf85315 None None] MFA Rules not processed for user `0914dada5c70437ba49688789eb31ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:35:40.084373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-888dd083-a510-4f5f-a774-15611cf85315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:40.184065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1332/2659] 10.222.0.16 () {58 vars in 1018 bytes} [Thu Jul 7 10:35:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:35:40.196372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c3e77-fd0a-4744-bf84-28d404e09dd6 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:35:40.198203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-287c3e77-fd0a-4744-bf84-28d404e09dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:40.265383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287c3e77-fd0a-4744-bf84-28d404e09dd6 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:35:40.269557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4b320f5e6fc4489484fe6599f4391245', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:35:40.270280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:40.270661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:40.271036 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:40.282483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:35:40.283980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:35:40.295731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1328/2660] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:35:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:35:42.032753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f878df-9660-446b-bab1-4b1767191144 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:42.033264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f878df-9660-446b-bab1-4b1767191144 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:42.033368 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f878df-9660-446b-bab1-4b1767191144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:42.051296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-98f878df-9660-446b-bab1-4b1767191144 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:35:42.052553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-98f878df-9660-446b-bab1-4b1767191144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:42.059604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1333/2661] 10.222.0.16 () {58 vars in 1009 bytes} [Thu Jul 7 10:35:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:35:42.071537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:35:42.082631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f5836-54b8-43e7-9bca-e6bc5e32242e service glance] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:35:42.088130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:35:42.088762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:42.089042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:42.089319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:42.097309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:35:42.098867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:35:42.107370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1329/2662] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:35:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:35:49.680543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f878df-9660-446b-bab1-4b1767191144 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:35:49.681951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-98f878df-9660-446b-bab1-4b1767191144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:49.743946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f878df-9660-446b-bab1-4b1767191144 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:35:49.747591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1604d79d-56c3-4669-b706-d6d527cb4a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:35:49.748291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604d79d-56c3-4669-b706-d6d527cb4a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:49.748573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604d79d-56c3-4669-b706-d6d527cb4a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:49.748842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1604d79d-56c3-4669-b706-d6d527cb4a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:49.750638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1604d79d-56c3-4669-b706-d6d527cb4a85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:49.765275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604d79d-56c3-4669-b706-d6d527cb4a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:35:49.766712 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1604d79d-56c3-4669-b706-d6d527cb4a85 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:35:49.777953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1334/2663] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:35:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:35:50.958596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:35:50.960756 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:50.972633 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ae5b3-e4eb-4a5e-99c6-88f783713680 service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:35:50.976786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:35:50.977536 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:50.977818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:50.978092 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:50.979787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:50.993202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:35:50.994634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:35:51.005642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1330/2664] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:35:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:35:58.487793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5b0d3-4779-47a0-a833-32067776b2d7 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:58.488843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5b0d3-4779-47a0-a833-32067776b2d7 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:58.489162 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b5b0d3-4779-47a0-a833-32067776b2d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:58.509079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-d9b5b0d3-4779-47a0-a833-32067776b2d7 None None] MFA Rules not processed for user `e74571f351a44ef48c8f51695a4e42d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:35:58.510487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-d9b5b0d3-4779-47a0-a833-32067776b2d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:35:58.518428 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1335/2665] 10.222.0.16 () {58 vars in 967 bytes} [Thu Jul 7 10:35:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:35:58.533874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:35:58.545539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485f2c5-df3e-45d8-afb8-c12e38186e91 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:35:58.551295 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadde869-9429-4943-8407-40936e91761e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e2e080fd7144086ac2cf5628a7a68ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:35:58.551978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadde869-9429-4943-8407-40936e91761e service swift] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:35:58.552267 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadde869-9429-4943-8407-40936e91761e service swift] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:35:58.552759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadde869-9429-4943-8407-40936e91761e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:35:58.561135 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadde869-9429-4943-8407-40936e91761e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:35:58.562745 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadde869-9429-4943-8407-40936e91761e service swift] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:35:58.571621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1331/2666] 10.222.0.16 () {60 vars in 1384 bytes} [Thu Jul 7 10:35:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 07 10:36:01.631996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:01.632434 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:01.632560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:01.649711 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] MFA Rules not processed for user `1222d8bb50554d64ac674f109b6ad077`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:01.650893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:01.657653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1336/2667] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:08.568372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=44 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:11.644613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=66 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:32.723510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadde869-9429-4943-8407-40936e91761e service swift] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:32.733063 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadde869-9429-4943-8407-40936e91761e service swift] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:32.736642 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:32.737277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:32.737562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:32.737842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:32.740130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:32.750896 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:32.752356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:32.761801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1332/2668] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:32 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:35.537136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:35.542226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:35.566407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c71891d-0487-4eb9-b545-54298c1803b9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:35.574444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:35.575430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:35.575943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:35.576406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:35.578510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:35.595272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:35.597256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:35.614621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1337/2669] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:36.335447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:36.347662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad11734-9ed0-419f-a072-11d28aafa1e5 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:36.351704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:36.352407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:36.352693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:36.352973 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:36.355153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:36.367434 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:36.368941 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:36.377581 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1333/2670] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:36 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:40.753906 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:40.763880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0eb4e-1947-409a-97a4-9636f641093f service nova] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:40.767990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:40.768696 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:40.768929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:40.769209 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:40.771550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:40.833930 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:40.835364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:40.844798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1338/2671] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:44.691847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:44.693889 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:44.711804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea48010d-78f1-4435-8d85-e31ecb0bd889 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:44.717139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:44.717985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:44.718361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:44.718754 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:44.720682 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:44.734045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:44.735627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:44.744778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:44.746496 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:44.747914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1334/2672] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:36:44.759420 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64266f06-d36e-4c97-8646-ece4a782bf55 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:44.764092 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:44.764972 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:44.765527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:44.765805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] PATH_INFO: `/v3/users/31bcb5c27d8b4ae2b9a3207fff0eabfe` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:44.767003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31bcb5c27d8b4ae2b9a3207fff0eabfe)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:44.768995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:44.820606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31bcb5c27d8b4ae2b9a3207fff0eabfe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:44.828606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31bcb5c27d8b4ae2b9a3207fff0eabfe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:44.829409 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31bcb5c27d8b4ae2b9a3207fff0eabfe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:44.842863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31bcb5c27d8b4ae2b9a3207fff0eabfe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:44.850954 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f373aca1-af66-4e66-a99a-25799b64f400 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31bcb5c27d8b4ae2b9a3207fff0eabfe'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:44.857810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1339/2673] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:44 2022] DELETE /identity/v3/users/31bcb5c27d8b4ae2b9a3207fff0eabfe => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:45.060960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.072308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3e6dc-66c9-4026-ae2b-0f11a92d365c service placement] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.130530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.131307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.131590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.131897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] PATH_INFO: `/v3/projects/96923df6a03841498e3732c8504b6363` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.133054 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96923df6a03841498e3732c8504b6363)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.134938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.178891 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96923df6a03841498e3732c8504b6363', 'request_id': 'req-cec9619b-214b-4c86-919a-4b3d36568d88'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:45.184849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96923df6a03841498e3732c8504b6363', 'request_id': 'req-cec9619b-214b-4c86-919a-4b3d36568d88'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:45.193271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96923df6a03841498e3732c8504b6363', 'request_id': 'req-cec9619b-214b-4c86-919a-4b3d36568d88'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:45.203486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] The token cache is being invalidate because project 96923df6a03841498e3732c8504b6363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:45.203804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:45.204427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-cec9619b-214b-4c86-919a-4b3d36568d88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:45.205953 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1335/2674] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:45 2022] DELETE /identity/v3/projects/96923df6a03841498e3732c8504b6363 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:45.211666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e30f2-6638-48a3-b1bf-06cde644f576 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.211993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e30f2-6638-48a3-b1bf-06cde644f576 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.212279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e30f2-6638-48a3-b1bf-06cde644f576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.230364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1c5e30f2-6638-48a3-b1bf-06cde644f576 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:45.231593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c5e30f2-6638-48a3-b1bf-06cde644f576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:45.281803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:45.299342 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1340/2675] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:45.307350 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b275ca-0bfb-478d-a289-f36125068fa0 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.307639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b275ca-0bfb-478d-a289-f36125068fa0 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.307945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b275ca-0bfb-478d-a289-f36125068fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.327784 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-60b275ca-0bfb-478d-a289-f36125068fa0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:45.328999 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-60b275ca-0bfb-478d-a289-f36125068fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:45.338342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1336/2676] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:36:45.342222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e30f2-6638-48a3-b1bf-06cde644f576 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.351921 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e30f2-6638-48a3-b1bf-06cde644f576 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.358116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.358864 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.359359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.359819 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.360812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:36:45.361130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.362830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.379127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1341/2677] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:36:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:36:45.381000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b275ca-0bfb-478d-a289-f36125068fa0 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.392979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b275ca-0bfb-478d-a289-f36125068fa0 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.398325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.398980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.399261 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.399534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.400269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.402159 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.427098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1337/2678] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:45 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:45.431271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.441427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3cb3fb-9de9-4372-a395-91a85cfa2621 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.496771 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.497432 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.497720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.497997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.498693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.500622 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.521749 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1342/2679] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:45 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:45.525799 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.536742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312f55b1-baec-4600-87ca-5f985c9c2518 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.541334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.541985 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.542263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.542537 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.543248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.544872 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.555033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1338/2680] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:45.558784 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.568977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85180d39-883d-4514-b290-562a5e9fd284 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.573548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890a2c37-bc75-45e8-8dd4-41e7de00958a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.574205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a2c37-bc75-45e8-8dd4-41e7de00958a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.574490 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a2c37-bc75-45e8-8dd4-41e7de00958a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.574767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890a2c37-bc75-45e8-8dd4-41e7de00958a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.575488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a2c37-bc75-45e8-8dd4-41e7de00958a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.577106 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890a2c37-bc75-45e8-8dd4-41e7de00958a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.587322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1343/2681] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:45.591427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.601902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b2a057-6bf3-41a9-a842-030f43797ee2 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.607085 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.608698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.609195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.609705 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] PATH_INFO: `/v3/projects/e1ec4c8aabe040eea34d18027fe0f2af/users/71426d1f2b114798a9e6166c555dc36f/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.621977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:45.630622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ec4c8aabe040eea34d18027fe0f2af, user_id=71426d1f2b114798a9e6166c555dc36f, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.633558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.646310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1339/2682] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:45 2022] PUT /identity/v3/projects/e1ec4c8aabe040eea34d18027fe0f2af/users/71426d1f2b114798a9e6166c555dc36f/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:45.656100 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2851-4105-4f93-96d9-335273e8cf7e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.656424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2851-4105-4f93-96d9-335273e8cf7e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.656705 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aa2851-4105-4f93-96d9-335273e8cf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.684380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-48aa2851-4105-4f93-96d9-335273e8cf7e None None] MFA Rules not processed for user `71426d1f2b114798a9e6166c555dc36f`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:45.685552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-48aa2851-4105-4f93-96d9-335273e8cf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:45.758072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1344/2683] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:45.762538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.827127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d893f2e5-6c79-4965-aff8-c581d35ff742 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.832088 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.833464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.833818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.834164 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.835014 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.837002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.862426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1340/2684] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:45 2022] POST /identity/v3/projects => generated 371 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:45.866895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2851-4105-4f93-96d9-335273e8cf7e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.877623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aa2851-4105-4f93-96d9-335273e8cf7e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.926227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:45.938851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.939727 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.940034 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.940308 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.941007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.943441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.966427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1345/2685] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:45 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:45.967640 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:45.978232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd694b1d-0d94-4ab3-8145-00cf378ec657 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:45.983203 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:45.983926 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:45.984202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:45.984475 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:45.985195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:45.986780 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:45.997706 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1341/2686] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:46.001945 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:46.012608 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e34a09-06c7-4fe4-acb4-5b82be6eb22b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:46.017261 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:46.018152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:46.018433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:46.018707 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] PATH_INFO: `/v3/projects/3645a76dac094995a40659671fe8a612/users/c02e415ac42e49c485714aeac11d2c51/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:46.034484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3645a76dac094995a40659671fe8a612, user_id=c02e415ac42e49c485714aeac11d2c51, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:46.037300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:46.053148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1346/2687] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:46 2022] PUT /identity/v3/projects/3645a76dac094995a40659671fe8a612/users/c02e415ac42e49c485714aeac11d2c51/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:46.057128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:46.068411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f35d0a2-0f6f-431a-8a47-5f1c4085dd8a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:46.137410 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9472c7-c2f3-4709-8c32-b911b77dcedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:46.138146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9472c7-c2f3-4709-8c32-b911b77dcedd admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:46.138382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9472c7-c2f3-4709-8c32-b911b77dcedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:46.138693 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9472c7-c2f3-4709-8c32-b911b77dcedd admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:46.139419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9472c7-c2f3-4709-8c32-b911b77dcedd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:46.141099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9472c7-c2f3-4709-8c32-b911b77dcedd admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:46.151623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1342/2688] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:46 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:46.155630 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:46.166338 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690fa24-375e-404a-a8aa-85a6221f5f70 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:46.174738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:46.174738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:46.174738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:46.174738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] PATH_INFO: `/v3/projects/3645a76dac094995a40659671fe8a612/users/c02e415ac42e49c485714aeac11d2c51/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:46.178175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] RBAC: Authorizing `identity:create_grant(project_id=3645a76dac094995a40659671fe8a612, user_id=c02e415ac42e49c485714aeac11d2c51, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:46.181033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:46.194343 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1347/2689] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:46 2022] PUT /identity/v3/projects/3645a76dac094995a40659671fe8a612/users/c02e415ac42e49c485714aeac11d2c51/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:46.207804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593a28-e57d-4e11-899d-38f41bf93da3 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:46.207804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593a28-e57d-4e11-899d-38f41bf93da3 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:46.207804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14593a28-e57d-4e11-899d-38f41bf93da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:46.232448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-14593a28-e57d-4e11-899d-38f41bf93da3 None None] MFA Rules not processed for user `c02e415ac42e49c485714aeac11d2c51`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:46.233646 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-14593a28-e57d-4e11-899d-38f41bf93da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:46.281631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:46.309849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1343/2690] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:46 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:46.322030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:46.323792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:46.380546 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ebdfda-32dd-4b86-b20e-5b68429e395c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:46.384188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:46.384834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:46.385123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:46.385452 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:46.394523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:46.396044 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:46.405460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1348/2691] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:46 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:46.475692 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593a28-e57d-4e11-899d-38f41bf93da3 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:46.485367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14593a28-e57d-4e11-899d-38f41bf93da3 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:46.489574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09addddc-6f6c-4d92-9595-71421473ed92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:46.490212 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09addddc-6f6c-4d92-9595-71421473ed92 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:46.490497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09addddc-6f6c-4d92-9595-71421473ed92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:46.490775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09addddc-6f6c-4d92-9595-71421473ed92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:46.547565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09addddc-6f6c-4d92-9595-71421473ed92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:46.549040 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09addddc-6f6c-4d92-9595-71421473ed92 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:46.558492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1344/2692] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:47.209066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:47.270123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e44e8a-88db-4716-a1d6-d22bb91e9570 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:47.275194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:47.276089 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:47.276402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:47.276680 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] PATH_INFO: `/v3/users/71426d1f2b114798a9e6166c555dc36f` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:47.277848 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] RBAC: Authorizing `identity:delete_user(user_id=71426d1f2b114798a9e6166c555dc36f)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:47.279776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:47.323017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:47.325009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71426d1f2b114798a9e6166c555dc36f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:47.333849 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71426d1f2b114798a9e6166c555dc36f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:47.334789 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71426d1f2b114798a9e6166c555dc36f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:47.347754 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71426d1f2b114798a9e6166c555dc36f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:47.354549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71426d1f2b114798a9e6166c555dc36f'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:47.362439 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1349/2693] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:47 2022] DELETE /identity/v3/users/71426d1f2b114798a9e6166c555dc36f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:47.366888 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:47.367391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:47.367508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:47.384776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:47.385975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:47.446198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1345/2694] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:47.455994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:47.457423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:47.540376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d10e5ef-2bae-44ea-afe7-092016f4b03f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:47.540376 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:47.541148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:47.541148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:47.541659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:47.553686 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:47.555869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:47.564882 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:47.566203 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1350/2695] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:47.566343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:47.579248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc8cdc-baf1-4237-8ba9-7ca99d9ea4eb None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:47.582849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:47.583529 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:47.583849 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:47.584126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:47.585937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:47.597832 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:47.599374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:47.609188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1346/2696] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:36:47.975721 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:47.977326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:47.990715 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff299a8-e5a5-4a97-abdf-6510e98424ac service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:47.996149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:47.997119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:47.997346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:47.997621 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] PATH_INFO: `/v3/projects/e1ec4c8aabe040eea34d18027fe0f2af` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:47.999553 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ec4c8aabe040eea34d18027fe0f2af)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:48.001508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:48.050969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ec4c8aabe040eea34d18027fe0f2af', 'request_id': 'req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.059866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ec4c8aabe040eea34d18027fe0f2af', 'request_id': 'req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.066488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ec4c8aabe040eea34d18027fe0f2af', 'request_id': 'req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.077055 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] The token cache is being invalidate because project e1ec4c8aabe040eea34d18027fe0f2af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:48.077356 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.078008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.082276 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1351/2697] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:47 2022] DELETE /identity/v3/projects/e1ec4c8aabe040eea34d18027fe0f2af => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:48.083595 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:48.093227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e839e-4172-422a-9705-fb6ac89a5206 service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:48.152291 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:48.153168 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:48.153452 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:48.153729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] PATH_INFO: `/v3/users/c02e415ac42e49c485714aeac11d2c51` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:48.154906 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c02e415ac42e49c485714aeac11d2c51)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:48.156963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:48.167791 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=100 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:48.206140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c02e415ac42e49c485714aeac11d2c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.214099 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c02e415ac42e49c485714aeac11d2c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.214716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c02e415ac42e49c485714aeac11d2c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.222540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c02e415ac42e49c485714aeac11d2c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.234961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-421263e9-5b99-4db7-a405-1706526ae4d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c02e415ac42e49c485714aeac11d2c51'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.245571 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1347/2698] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:48 2022] DELETE /identity/v3/users/c02e415ac42e49c485714aeac11d2c51 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:48.659066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:48.662170 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:48.674588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7a777b-95d5-41c9-8822-1a82e2a3a05b admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:48.732788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:48.746013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:48.746913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:48.747593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:48.747913 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] PATH_INFO: `/v3/projects/3645a76dac094995a40659671fe8a612` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:48.749332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3645a76dac094995a40659671fe8a612)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:48.751393 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:48.809462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3645a76dac094995a40659671fe8a612', 'request_id': 'req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.820216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3645a76dac094995a40659671fe8a612', 'request_id': 'req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.827628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3645a76dac094995a40659671fe8a612', 'request_id': 'req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.839851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] The token cache is being invalidate because project 3645a76dac094995a40659671fe8a612 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:48.840183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.840794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-3d06b73d-a0ab-42ef-9918-bc880dcc1a96 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:48.843798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1352/2699] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:48 2022] DELETE /identity/v3/projects/3645a76dac094995a40659671fe8a612 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:48.847289 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95840773-fd9b-4a8c-9421-29ba7af5f0ac None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:48.847847 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95840773-fd9b-4a8c-9421-29ba7af5f0ac None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:48.848003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95840773-fd9b-4a8c-9421-29ba7af5f0ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:48.868156 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-95840773-fd9b-4a8c-9421-29ba7af5f0ac None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:48.869386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-95840773-fd9b-4a8c-9421-29ba7af5f0ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:48.943332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1348/2700] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:48.951306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f575bb-de2f-485f-9aba-fb2791f012c5 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:48.951628 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f575bb-de2f-485f-9aba-fb2791f012c5 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:48.951949 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f575bb-de2f-485f-9aba-fb2791f012c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:48.970917 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-23f575bb-de2f-485f-9aba-fb2791f012c5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:48.972108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-23f575bb-de2f-485f-9aba-fb2791f012c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:48.980122 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1353/2701] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:48.984416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95840773-fd9b-4a8c-9421-29ba7af5f0ac None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:48.994333 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95840773-fd9b-4a8c-9421-29ba7af5f0ac None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:48.998923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:48.999677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.000080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.000366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.001091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:36:49.001401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.002994 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.015143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1349/2702] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:36:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:36:49.019260 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f575bb-de2f-485f-9aba-fb2791f012c5 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.029082 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f575bb-de2f-485f-9aba-fb2791f012c5 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.034872 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.035523 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.035836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.036116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.036805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.038760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.065069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1354/2703] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:49 2022] POST /identity/v3/projects => generated 363 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:49.069635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.079718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49737626-4b1a-4148-9748-b010f948aae4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.138431 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.139463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.139764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.140172 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.140911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.142874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.166438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1350/2704] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:49 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:49.167803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.179030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0c89ff-4e55-43e7-a985-fac79577bf7f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.183732 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.184437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.184719 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.184992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.185725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.187367 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.199484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1355/2705] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:49.203662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.210771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:49.215110 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7798b-c29f-488b-b5ea-7ae3ee082a24 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.222149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82680042-b6a3-47d1-b552-c2b9a337d393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.223007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82680042-b6a3-47d1-b552-c2b9a337d393 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.223464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82680042-b6a3-47d1-b552-c2b9a337d393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.224018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82680042-b6a3-47d1-b552-c2b9a337d393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.224805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82680042-b6a3-47d1-b552-c2b9a337d393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.226448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82680042-b6a3-47d1-b552-c2b9a337d393 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.240980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1351/2706] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:49.242223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.252669 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1a9b00-14dc-4bdc-8527-51cb07790377 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.258269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.259184 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.259467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.259761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] PATH_INFO: `/v3/projects/5196d680636e45e7bf39be7d7043ca77/users/da1670c50cc14874a756f95de35f9e9a/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.275483 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=5196d680636e45e7bf39be7d7043ca77, user_id=da1670c50cc14874a756f95de35f9e9a, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.278192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.293091 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1356/2707] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:49 2022] PUT /identity/v3/projects/5196d680636e45e7bf39be7d7043ca77/users/da1670c50cc14874a756f95de35f9e9a/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:49.303963 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270e2e5-18c6-4ca0-8bcf-46f9f6a04500 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.304340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270e2e5-18c6-4ca0-8bcf-46f9f6a04500 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.304681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270e2e5-18c6-4ca0-8bcf-46f9f6a04500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.334283 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3270e2e5-18c6-4ca0-8bcf-46f9f6a04500 None None] MFA Rules not processed for user `da1670c50cc14874a756f95de35f9e9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:49.335630 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3270e2e5-18c6-4ca0-8bcf-46f9f6a04500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:49.402316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1352/2708] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:49 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:49.406910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.471976 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde879a8-5022-4688-a829-d96f0ec057cd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.477444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.478198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.478579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.478946 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.479720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.481720 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.508875 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:49.528210 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270e2e5-18c6-4ca0-8bcf-46f9f6a04500 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.530101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1357/2709] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:49 2022] POST /identity/v3/projects => generated 363 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:49.541770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270e2e5-18c6-4ca0-8bcf-46f9f6a04500 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.605337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.606268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.606622 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.606974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.607767 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.610007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.641825 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1353/2710] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:49 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:36:49.643815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.661880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452318b0-603b-45dc-b9b7-ca80844f24f7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.668223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.669038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.669500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.670300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.671067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.673069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.692481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1358/2711] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:49.697438 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.715597 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c2cbd-19fd-4f46-8215-231b7300781b admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.722726 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.724474 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.724862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.725240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] PATH_INFO: `/v3/projects/1434c226dceb45b6ba6696ad7a5d9e3e/users/5ed424d85a624aa5894fa7f7fc0f5788/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.745828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1434c226dceb45b6ba6696ad7a5d9e3e, user_id=5ed424d85a624aa5894fa7f7fc0f5788, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.748865 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.769809 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1354/2712] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:49 2022] PUT /identity/v3/projects/1434c226dceb45b6ba6696ad7a5d9e3e/users/5ed424d85a624aa5894fa7f7fc0f5788/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:49.775651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.791373 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfe1c7-b4e0-4501-b36f-d22a171105da admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.856867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d6f26-7311-4d69-96ca-752a1200e3b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.857685 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d6f26-7311-4d69-96ca-752a1200e3b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.857989 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d6f26-7311-4d69-96ca-752a1200e3b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.858788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d6f26-7311-4d69-96ca-752a1200e3b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.858993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d6f26-7311-4d69-96ca-752a1200e3b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.860667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d6f26-7311-4d69-96ca-752a1200e3b8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.874353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1359/2713] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:49 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:49.875533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:49.886699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b4706-efa8-47a2-8783-7c4be9e1975d admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:49.891942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:49.893067 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.893527 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.893771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] PATH_INFO: `/v3/projects/1434c226dceb45b6ba6696ad7a5d9e3e/users/5ed424d85a624aa5894fa7f7fc0f5788/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.896858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1434c226dceb45b6ba6696ad7a5d9e3e, user_id=5ed424d85a624aa5894fa7f7fc0f5788, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:49.900389 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:49.917678 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1355/2714] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:49 2022] PUT /identity/v3/projects/1434c226dceb45b6ba6696ad7a5d9e3e/users/5ed424d85a624aa5894fa7f7fc0f5788/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:49.923702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e2df-ce73-4ce8-88fa-e59731599d2f None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:49.924008 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e2df-ce73-4ce8-88fa-e59731599d2f None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:49.924299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6371e2df-ce73-4ce8-88fa-e59731599d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:49.952758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6371e2df-ce73-4ce8-88fa-e59731599d2f None None] MFA Rules not processed for user `5ed424d85a624aa5894fa7f7fc0f5788`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:49.954141 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6371e2df-ce73-4ce8-88fa-e59731599d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:50.030238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1360/2715] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:49 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:36:50.043338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:50.045121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:50.094516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:50.119380 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb20878f-3621-4031-b106-f3d5bfe135e4 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:50.123427 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fde1d8-b0f1-40f0-9133-0c744b82e465 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:50.124146 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde1d8-b0f1-40f0-9133-0c744b82e465 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:50.124857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde1d8-b0f1-40f0-9133-0c744b82e465 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:50.124857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fde1d8-b0f1-40f0-9133-0c744b82e465 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:50.137753 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fde1d8-b0f1-40f0-9133-0c744b82e465 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:50.139235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fde1d8-b0f1-40f0-9133-0c744b82e465 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:50.160738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1356/2716] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:50 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:50.644135 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e2df-ce73-4ce8-88fa-e59731599d2f None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:50.718753 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6371e2df-ce73-4ce8-88fa-e59731599d2f None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:50.723572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:50.724515 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:50.724805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:50.725211 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] PATH_INFO: `/v3/users/da1670c50cc14874a756f95de35f9e9a` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:50.726510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da1670c50cc14874a756f95de35f9e9a)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:50.728539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:50.741697 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:50.775396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da1670c50cc14874a756f95de35f9e9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:50.782863 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da1670c50cc14874a756f95de35f9e9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:50.783383 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da1670c50cc14874a756f95de35f9e9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:50.795575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da1670c50cc14874a756f95de35f9e9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:50.803213 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da1670c50cc14874a756f95de35f9e9a'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:50.809239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1361/2717] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:50 2022] DELETE /identity/v3/users/da1670c50cc14874a756f95de35f9e9a => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:50.813868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:50.814236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:50.814830 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:50.832801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:50.834083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:50.898000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1357/2718] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:50.907477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:50.908914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:50.992304 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572964d4-1290-4e3d-81b3-cc5290466824 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:50.997835 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:50.998503 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:50.998791 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:50.999068 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:51.010606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:51.012097 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:51.022760 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1362/2719] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:51.346381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:51.348027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:51.368565 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82d018b-4338-4c33-bfe3-401b797c5b41 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:51.375871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:51.377486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:51.378917 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:51.379604 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] PATH_INFO: `/v3/projects/5196d680636e45e7bf39be7d7043ca77` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:51.382684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5196d680636e45e7bf39be7d7043ca77)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:51.384903 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:51.446355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5196d680636e45e7bf39be7d7043ca77', 'request_id': 'req-1e66c62e-706d-4183-9819-afb3a03a4212'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.458794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5196d680636e45e7bf39be7d7043ca77', 'request_id': 'req-1e66c62e-706d-4183-9819-afb3a03a4212'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.461073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5196d680636e45e7bf39be7d7043ca77', 'request_id': 'req-1e66c62e-706d-4183-9819-afb3a03a4212'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.472090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] The token cache is being invalidate because project 5196d680636e45e7bf39be7d7043ca77 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:51.472380 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.473024 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.475280 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1358/2720] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:51 2022] DELETE /identity/v3/projects/5196d680636e45e7bf39be7d7043ca77 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:51.479612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:51.489349 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce5484-10c4-4c58-9f66-0e3b3f44f321 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:51.553616 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:51.554493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:51.554772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:51.555043 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] PATH_INFO: `/v3/users/5ed424d85a624aa5894fa7f7fc0f5788` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:51.556282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ed424d85a624aa5894fa7f7fc0f5788)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:51.558194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:51.596445 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:51.607968 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ed424d85a624aa5894fa7f7fc0f5788'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.616684 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ed424d85a624aa5894fa7f7fc0f5788'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.617295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ed424d85a624aa5894fa7f7fc0f5788'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.630651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ed424d85a624aa5894fa7f7fc0f5788'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.637761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-df7b2f44-d1b5-456a-aa1f-8216a7d33e45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ed424d85a624aa5894fa7f7fc0f5788'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:51.644056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1363/2721] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:51 2022] DELETE /identity/v3/users/5ed424d85a624aa5894fa7f7fc0f5788 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:52.092661 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.094090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:52.106942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e66c62e-706d-4183-9819-afb3a03a4212 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.167601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.168412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.168691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.168961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] PATH_INFO: `/v3/projects/1434c226dceb45b6ba6696ad7a5d9e3e` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.170158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] RBAC: Authorizing `identity:delete_project(project_id=1434c226dceb45b6ba6696ad7a5d9e3e)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.172119 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.174492 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:52.242674 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1434c226dceb45b6ba6696ad7a5d9e3e', 'request_id': 'req-fd1a9399-9357-448b-b9dc-109add8335db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:52.250770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1434c226dceb45b6ba6696ad7a5d9e3e', 'request_id': 'req-fd1a9399-9357-448b-b9dc-109add8335db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:52.260397 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1434c226dceb45b6ba6696ad7a5d9e3e', 'request_id': 'req-fd1a9399-9357-448b-b9dc-109add8335db'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:52.272639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] The token cache is being invalidate because project 1434c226dceb45b6ba6696ad7a5d9e3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:52.273059 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:52.273808 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-fd1a9399-9357-448b-b9dc-109add8335db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:52.275730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1359/2722] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:52 2022] DELETE /identity/v3/projects/1434c226dceb45b6ba6696ad7a5d9e3e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:52.280691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10030334-4ab0-4a56-88c5-9c0eff7d4519 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.281249 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10030334-4ab0-4a56-88c5-9c0eff7d4519 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.281364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10030334-4ab0-4a56-88c5-9c0eff7d4519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.305623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-10030334-4ab0-4a56-88c5-9c0eff7d4519 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:52.306853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-10030334-4ab0-4a56-88c5-9c0eff7d4519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:52.387793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1364/2723] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:52.393623 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f534-11c4-4607-a20c-812902f2cc84 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.394201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f534-11c4-4607-a20c-812902f2cc84 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.394538 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c0f534-11c4-4607-a20c-812902f2cc84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.414868 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-b0c0f534-11c4-4607-a20c-812902f2cc84 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:52.416129 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-b0c0f534-11c4-4607-a20c-812902f2cc84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:52.425006 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1360/2724] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:52.430847 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10030334-4ab0-4a56-88c5-9c0eff7d4519 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.444239 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10030334-4ab0-4a56-88c5-9c0eff7d4519 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.448941 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.449643 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.449934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.450223 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.450986 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:36:52.451416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.453069 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.467776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1365/2725] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:36:52.470035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f534-11c4-4607-a20c-812902f2cc84 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.480515 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c0f534-11c4-4607-a20c-812902f2cc84 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.485383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.486077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.486348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.486621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.487340 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.489308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.529134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1361/2726] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:52 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:52.530508 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.541710 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a413e892-e0b0-4237-85bf-5458374f3878 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.605018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.605709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.605992 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.606269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.606973 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.608948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.631024 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1366/2727] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:52 2022] POST /identity/v3/users => generated 285 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:52.635182 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.646673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be385aea-d0b4-449b-9fb7-e50d068d59d0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.651594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.652316 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.652603 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.652878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.653602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.655196 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.666273 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1362/2728] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:52.670318 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.680482 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f573dfc-735d-4758-86f5-2504422b2fb6 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.685096 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500dde4-eb0e-4e51-a144-23fca4c61fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.685776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dde4-eb0e-4e51-a144-23fca4c61fc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.686059 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dde4-eb0e-4e51-a144-23fca4c61fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.686335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500dde4-eb0e-4e51-a144-23fca4c61fc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.687066 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500dde4-eb0e-4e51-a144-23fca4c61fc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.688706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500dde4-eb0e-4e51-a144-23fca4c61fc3 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.700176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1367/2729] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:52.704446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.715325 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163eff4-e0cd-4007-b166-092a93d6783f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.720995 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.722000 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.722286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.722570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] PATH_INFO: `/v3/projects/7eac9152aa5b445a888b86eeffa5c8aa/users/1acdd9b2a15c49588e85f282df2b5515/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.737987 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7eac9152aa5b445a888b86eeffa5c8aa, user_id=1acdd9b2a15c49588e85f282df2b5515, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.740680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.753968 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1363/2730] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:52 2022] PUT /identity/v3/projects/7eac9152aa5b445a888b86eeffa5c8aa/users/1acdd9b2a15c49588e85f282df2b5515/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:52.765187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94f795-3fdd-44b0-a4eb-9ae80f460fd1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.765484 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94f795-3fdd-44b0-a4eb-9ae80f460fd1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.765769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94f795-3fdd-44b0-a4eb-9ae80f460fd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.794058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-df94f795-3fdd-44b0-a4eb-9ae80f460fd1 None None] MFA Rules not processed for user `1acdd9b2a15c49588e85f282df2b5515`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:52.795287 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-df94f795-3fdd-44b0-a4eb-9ae80f460fd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:52.800967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:52.865224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1368/2731] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:52 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:52.869752 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.932617 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5213846-feaf-4c27-ad4c-b7dd8c335fe7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:52.937341 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:52.937996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:52.938279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:52.938558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:52.939240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:52.941453 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:52.971643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1364/2732] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:52 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:52.975678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94f795-3fdd-44b0-a4eb-9ae80f460fd1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:52.986865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94f795-3fdd-44b0-a4eb-9ae80f460fd1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:53.073322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:53.074225 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:53.074834 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:53.075282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:53.076191 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:53.078424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:53.106772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1369/2733] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:52 2022] POST /identity/v3/users => generated 290 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:36:53.108087 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:53.121838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6760dc64-1a19-46f8-a128-7828addab24a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:53.128115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:53.128996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:53.129508 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:53.129939 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:53.131516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:53.133323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:53.147574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1365/2734] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:53.151981 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:53.163728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3022adc-2601-4fe5-abbe-8eada744ccc1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:53.168793 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:53.169785 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:53.170180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:53.170571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] PATH_INFO: `/v3/projects/c9f3df27a98f4098b9f5195129cdae53/users/2e6193a52c8940f4b613538ed4bd07e1/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:53.196598 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9f3df27a98f4098b9f5195129cdae53, user_id=2e6193a52c8940f4b613538ed4bd07e1, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:53.199810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:53.218345 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1370/2735] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:53 2022] PUT /identity/v3/projects/c9f3df27a98f4098b9f5195129cdae53/users/2e6193a52c8940f4b613538ed4bd07e1/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:53.223132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:53.238221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:53.251149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4407aae-c659-4df4-ba25-d8c0b90a4809 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:53.350525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d79ed89-034c-4ab1-af95-f46f0622e64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:53.351304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ed89-034c-4ab1-af95-f46f0622e64d admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:53.351664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ed89-034c-4ab1-af95-f46f0622e64d admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:53.352037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79ed89-034c-4ab1-af95-f46f0622e64d admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:53.352966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79ed89-034c-4ab1-af95-f46f0622e64d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:53.356116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d79ed89-034c-4ab1-af95-f46f0622e64d admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:53.372218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1366/2736] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:53 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:53.376841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:53.393348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5246e2e7-b817-4994-8c44-aa2695d4ed7f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:53.403280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:53.404827 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:53.405123 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:53.405538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] PATH_INFO: `/v3/projects/c9f3df27a98f4098b9f5195129cdae53/users/2e6193a52c8940f4b613538ed4bd07e1/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:53.408971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9f3df27a98f4098b9f5195129cdae53, user_id=2e6193a52c8940f4b613538ed4bd07e1, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:53.413850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:53.445735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1371/2737] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:53 2022] PUT /identity/v3/projects/c9f3df27a98f4098b9f5195129cdae53/users/2e6193a52c8940f4b613538ed4bd07e1/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:53.514238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:53.514514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:53.514734 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:53.566147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] MFA Rules not processed for user `2e6193a52c8940f4b613538ed4bd07e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:53.567459 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:53.674691 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1367/2738] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:53 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:53.686646 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:53.688661 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:53.748073 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfef8593-6a23-4c2f-ad69-6f2bd9c10aa7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:53.751609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:53.752289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:53.752575 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:53.752853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:53.762198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:53.763851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:53.774128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1372/2739] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:53 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:54.787093 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:54.788575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:54.883345 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c3b22-0859-44a9-9218-47869a811fc5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:54.888699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:54.889624 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:54.890028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:54.890202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] PATH_INFO: `/v3/users/059a87a80fc547db8aeb3bb03003cd2c` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:54.891368 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=059a87a80fc547db8aeb3bb03003cd2c)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:54.893304 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:54.923641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:54.946913 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '059a87a80fc547db8aeb3bb03003cd2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:54.956367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '059a87a80fc547db8aeb3bb03003cd2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:54.957112 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '059a87a80fc547db8aeb3bb03003cd2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:54.964662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '059a87a80fc547db8aeb3bb03003cd2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:54.984107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '059a87a80fc547db8aeb3bb03003cd2c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:54.994602 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1368/2740] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:54 2022] DELETE /identity/v3/users/059a87a80fc547db8aeb3bb03003cd2c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:55.175398 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:55.180769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:55.189583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae90618-4b49-45d1-a0f6-6a7bc3787083 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:55.261419 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:55.262362 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:55.262554 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:55.262855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] PATH_INFO: `/v3/projects/12a91d56e163442dacf4c5ce7b28a132` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:55.264139 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] RBAC: Authorizing `identity:delete_project(project_id=12a91d56e163442dacf4c5ce7b28a132)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:55.266045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:55.316634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:55.318143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:55.318295 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12a91d56e163442dacf4c5ce7b28a132', 'request_id': 'req-db4a97b3-3e45-42c1-a9b6-cb165248233e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.325583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12a91d56e163442dacf4c5ce7b28a132', 'request_id': 'req-db4a97b3-3e45-42c1-a9b6-cb165248233e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.332426 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12a91d56e163442dacf4c5ce7b28a132', 'request_id': 'req-db4a97b3-3e45-42c1-a9b6-cb165248233e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.332673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbae873-b3ae-4205-9a1d-ba98873ed1a6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:55.344346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] The token cache is being invalidate because project 12a91d56e163442dacf4c5ce7b28a132 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:55.344638 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.345286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.350631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1373/2741] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:55 2022] DELETE /identity/v3/projects/12a91d56e163442dacf4c5ce7b28a132 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:55.351463 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:55.352807 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:55.394396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:55.395279 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:55.395540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:55.396367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] PATH_INFO: `/v3/users/1acdd9b2a15c49588e85f282df2b5515` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:55.397984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=1acdd9b2a15c49588e85f282df2b5515)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:55.399979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:55.420543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a97b3-3e45-42c1-a9b6-cb165248233e admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:55.426166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:55.427513 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:55.427853 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:55.428118 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] PATH_INFO: `/v3/users/49b589a8aea6457d8a53d811d9b6fd16` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:55.429519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b589a8aea6457d8a53d811d9b6fd16)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:55.432391 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:55.448593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1acdd9b2a15c49588e85f282df2b5515'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.456950 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1acdd9b2a15c49588e85f282df2b5515'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.458125 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1acdd9b2a15c49588e85f282df2b5515'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.466740 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1acdd9b2a15c49588e85f282df2b5515'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.472527 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:55.479984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-51bb191b-b61a-4bb1-8a70-9f025f1a3fdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1acdd9b2a15c49588e85f282df2b5515'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.482038 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b589a8aea6457d8a53d811d9b6fd16'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.486153 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1369/2742] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:55 2022] DELETE /identity/v3/users/1acdd9b2a15c49588e85f282df2b5515 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:55.489818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b589a8aea6457d8a53d811d9b6fd16'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.490533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:55.490678 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b589a8aea6457d8a53d811d9b6fd16'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.490819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:55.491098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:55.502648 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b589a8aea6457d8a53d811d9b6fd16'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.509105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:55.510050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b589a8aea6457d8a53d811d9b6fd16'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.510456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:55.516322 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1374/2743] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:55 2022] DELETE /identity/v3/users/49b589a8aea6457d8a53d811d9b6fd16 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:55.528601 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:55.573500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1370/2744] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:36:55.583050 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:55.585028 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:55.645119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f6cce3-bb65-4a40-a283-b6afa8d49ce5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:55.649430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:55.658058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:55.658058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:55.658058 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:55.698064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:55.699483 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:55.712233 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4b73a6-71fc-4eef-8ef2-290d3a33dda5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:55.728767 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:55.729961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:55.741099 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1375/2745] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:55.777042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:55.777905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:55.778184 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:55.779068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] PATH_INFO: `/v3/projects/79d3d594053f4e62a2718c7ee83b8439` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:55.780337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] RBAC: Authorizing `identity:delete_project(project_id=79d3d594053f4e62a2718c7ee83b8439)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:55.782458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:55.833363 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79d3d594053f4e62a2718c7ee83b8439', 'request_id': 'req-1418e47c-36c4-415d-aa12-51c4a0d4e596'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.839838 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79d3d594053f4e62a2718c7ee83b8439', 'request_id': 'req-1418e47c-36c4-415d-aa12-51c4a0d4e596'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.848201 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79d3d594053f4e62a2718c7ee83b8439', 'request_id': 'req-1418e47c-36c4-415d-aa12-51c4a0d4e596'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.859167 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] The token cache is being invalidate because project 79d3d594053f4e62a2718c7ee83b8439 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:55.859454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.860149 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:55.861739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1371/2746] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:55 2022] DELETE /identity/v3/projects/79d3d594053f4e62a2718c7ee83b8439 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:56.029065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:56.030560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:56.048790 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9912b5ad-fe1c-421f-8037-9da52abc088a service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:56.111469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:56.112388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:56.112674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:56.112950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] PATH_INFO: `/v3/projects/7eac9152aa5b445a888b86eeffa5c8aa` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:56.114219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7eac9152aa5b445a888b86eeffa5c8aa)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:56.117410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:56.120109 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:56.165889 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7eac9152aa5b445a888b86eeffa5c8aa', 'request_id': 'req-e883a8b8-5892-48d9-ae58-5a2a7e896804'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.173788 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7eac9152aa5b445a888b86eeffa5c8aa', 'request_id': 'req-e883a8b8-5892-48d9-ae58-5a2a7e896804'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.180961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7eac9152aa5b445a888b86eeffa5c8aa', 'request_id': 'req-e883a8b8-5892-48d9-ae58-5a2a7e896804'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.191359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] The token cache is being invalidate because project 7eac9152aa5b445a888b86eeffa5c8aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:56.191654 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.192386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.193955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1376/2747] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:56 2022] DELETE /identity/v3/projects/7eac9152aa5b445a888b86eeffa5c8aa => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:56.197795 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:56.199199 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:56.211787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418e47c-36c4-415d-aa12-51c4a0d4e596 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:56.258725 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:56.268577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:56.269461 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:56.269933 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:56.270232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] PATH_INFO: `/v3/users/2e6193a52c8940f4b613538ed4bd07e1` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:56.271505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6193a52c8940f4b613538ed4bd07e1)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:56.273436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:56.318735 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6193a52c8940f4b613538ed4bd07e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.325356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6193a52c8940f4b613538ed4bd07e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.326170 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6193a52c8940f4b613538ed4bd07e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.332978 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6193a52c8940f4b613538ed4bd07e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.344398 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6a9fbf7a-d872-4b49-8f1e-2f2d2ef9af8b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6193a52c8940f4b613538ed4bd07e1'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.350627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1372/2748] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:56 2022] DELETE /identity/v3/users/2e6193a52c8940f4b613538ed4bd07e1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:56.611205 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:56.612682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:56.627228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883a8b8-5892-48d9-ae58-5a2a7e896804 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:56.684615 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:56.685481 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:56.685768 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:56.686040 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] PATH_INFO: `/v3/projects/c9f3df27a98f4098b9f5195129cdae53` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:56.687207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9f3df27a98f4098b9f5195129cdae53)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:56.689155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:56.736119 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9f3df27a98f4098b9f5195129cdae53', 'request_id': 'req-9e9ef79b-5000-4955-937b-09ae952b8a35'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.743150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9f3df27a98f4098b9f5195129cdae53', 'request_id': 'req-9e9ef79b-5000-4955-937b-09ae952b8a35'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.749271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9f3df27a98f4098b9f5195129cdae53', 'request_id': 'req-9e9ef79b-5000-4955-937b-09ae952b8a35'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.759436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] The token cache is being invalidate because project c9f3df27a98f4098b9f5195129cdae53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80142) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:56.759728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.760379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-9e9ef79b-5000-4955-937b-09ae952b8a35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:56.761919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1377/2749] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:56 2022] DELETE /identity/v3/projects/c9f3df27a98f4098b9f5195129cdae53 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:56.766742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dc0b2-1f64-4d7c-a1a3-18fb00b8bb56 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:56.767027 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dc0b2-1f64-4d7c-a1a3-18fb00b8bb56 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:56.767303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514dc0b2-1f64-4d7c-a1a3-18fb00b8bb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:56.784873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-514dc0b2-1f64-4d7c-a1a3-18fb00b8bb56 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:56.786064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-514dc0b2-1f64-4d7c-a1a3-18fb00b8bb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:56.844755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1373/2750] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:56.853127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f9297-071b-46cc-accb-64ed12456770 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:56.853416 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f9297-071b-46cc-accb-64ed12456770 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:56.853698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f9297-071b-46cc-accb-64ed12456770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:56.871467 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-584f9297-071b-46cc-accb-64ed12456770 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:56.872967 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-584f9297-071b-46cc-accb-64ed12456770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:56.881915 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1378/2751] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:56.885967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dc0b2-1f64-4d7c-a1a3-18fb00b8bb56 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:56.896611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514dc0b2-1f64-4d7c-a1a3-18fb00b8bb56 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:56.901236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:56.901890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:56.902177 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:56.902454 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] PATH_INFO: `/v3/domains` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:56.903178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80141) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:36:56.903488 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:56.905134 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:56.916665 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1374/2752] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:36:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:36:56.920769 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f9297-071b-46cc-accb-64ed12456770 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:56.929985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f9297-071b-46cc-accb-64ed12456770 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:56.934679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:56.935337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:56.935627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:56.935939 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:56.936645 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:56.938586 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:56.952725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:56.968442 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1379/2753] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:56 2022] POST /identity/v3/projects => generated 381 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:56.970757 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:56.980673 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572fd9a5-c622-4d94-a837-371ba8ec5899 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.035858 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.036549 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.036839 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.037126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.037801 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.039724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.046248 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:57.062323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1375/2754] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:56 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:57.066718 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.077526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9232ce-b951-48ae-af49-3d18145efdc9 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.082102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.082765 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.083080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.084007 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.084735 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.086326 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.096690 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1380/2755] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:57.100489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.110352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a943e-4bcf-4da2-a476-58125eaf16ab admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.114946 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7499ea68-7542-483b-aee6-2d591ec83913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.115591 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ea68-7542-483b-aee6-2d591ec83913 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.115910 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ea68-7542-483b-aee6-2d591ec83913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.116188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7499ea68-7542-483b-aee6-2d591ec83913 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.116895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499ea68-7542-483b-aee6-2d591ec83913 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.118472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7499ea68-7542-483b-aee6-2d591ec83913 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.128334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1376/2756] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:57.132752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.142516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf72758-4c11-4765-8bb1-fd9f420dba70 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.147631 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.148541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.148823 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.149095 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] PATH_INFO: `/v3/projects/4829655a00f943c48b02b08eaf91df1d/users/54ff6b4fdd34405fac4ff8aa11eaa80e/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.163544 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4829655a00f943c48b02b08eaf91df1d, user_id=54ff6b4fdd34405fac4ff8aa11eaa80e, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.166227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.179443 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1381/2757] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:57 2022] PUT /identity/v3/projects/4829655a00f943c48b02b08eaf91df1d/users/54ff6b4fdd34405fac4ff8aa11eaa80e/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:57.190741 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb4689-116a-4bae-b62e-a446fecb7962 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.191032 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb4689-116a-4bae-b62e-a446fecb7962 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.191328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb4689-116a-4bae-b62e-a446fecb7962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.218387 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-24eb4689-116a-4bae-b62e-a446fecb7962 None None] MFA Rules not processed for user `54ff6b4fdd34405fac4ff8aa11eaa80e`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:57.219592 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-24eb4689-116a-4bae-b62e-a446fecb7962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:57.281303 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1377/2758] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:57.285695 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.346869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75037a55-92d8-4027-822b-a3808d5799fd admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.351427 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.352124 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.352414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.352691 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.353430 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.355331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.382509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1382/2759] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:57 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:57.386286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb4689-116a-4bae-b62e-a446fecb7962 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.396967 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb4689-116a-4bae-b62e-a446fecb7962 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.455090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.455787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.456076 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.456361 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] PATH_INFO: `/v3/users` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.457058 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.458979 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.477916 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1378/2760] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:57 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:57.481854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.492560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de48ca-a019-4707-b4e3-8b49edc986fb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.497193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.497852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.498140 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.498424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.499149 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.500772 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.514802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1383/2761] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:57.516257 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.526230 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61aee1-70e9-4181-b330-0efbd525279e admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.531025 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.531977 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.532281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.532553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] PATH_INFO: `/v3/projects/c6cd903d52674ea1948082bda7e021a7/users/96935360874e4d79ae45138cbca040bb/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.548535 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cd903d52674ea1948082bda7e021a7, user_id=96935360874e4d79ae45138cbca040bb, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.551158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.564186 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1379/2762] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:57 2022] PUT /identity/v3/projects/c6cd903d52674ea1948082bda7e021a7/users/96935360874e4d79ae45138cbca040bb/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:57.569032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.578903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b6335f-ab2c-4ab9-9ca6-36def57b6cd7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.634676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370d7ea-fc28-431d-aaa1-e0b00c83d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.635266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d7ea-fc28-431d-aaa1-e0b00c83d84a admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.635597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d7ea-fc28-431d-aaa1-e0b00c83d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.636282 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370d7ea-fc28-431d-aaa1-e0b00c83d84a admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.637116 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370d7ea-fc28-431d-aaa1-e0b00c83d84a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.638805 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370d7ea-fc28-431d-aaa1-e0b00c83d84a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.648565 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1384/2763] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:57.652762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.663421 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5c74a7-83c9-4a08-ab50-f4bbea9e2b79 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.668090 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.668959 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.669232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.669525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] PATH_INFO: `/v3/projects/c6cd903d52674ea1948082bda7e021a7/users/96935360874e4d79ae45138cbca040bb/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.671445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cd903d52674ea1948082bda7e021a7, user_id=96935360874e4d79ae45138cbca040bb, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.674102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.684822 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1380/2764] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:57 2022] PUT /identity/v3/projects/c6cd903d52674ea1948082bda7e021a7/users/96935360874e4d79ae45138cbca040bb/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:57.695131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2cbd2-f8ed-4299-9dc9-87de7a932aa9 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.695423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2cbd2-f8ed-4299-9dc9-87de7a932aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.695703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb2cbd2-f8ed-4299-9dc9-87de7a932aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.723217 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-feb2cbd2-f8ed-4299-9dc9-87de7a932aa9 None None] MFA Rules not processed for user `96935360874e4d79ae45138cbca040bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:57.724994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-feb2cbd2-f8ed-4299-9dc9-87de7a932aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:57.742604 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:57.794374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1385/2765] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:57 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:57.805505 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:57.807198 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:57.829463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:57.868381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4dd20e-a60e-4b57-8bdc-0e3960479346 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:57.872044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950ec74-f38e-4ea8-a02e-b75726c317ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:57.872689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950ec74-f38e-4ea8-a02e-b75726c317ad service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:57.872971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950ec74-f38e-4ea8-a02e-b75726c317ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:57.873247 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950ec74-f38e-4ea8-a02e-b75726c317ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:57.882068 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950ec74-f38e-4ea8-a02e-b75726c317ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:57.883497 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950ec74-f38e-4ea8-a02e-b75726c317ad service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:57.892823 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1381/2766] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:58.237142 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2cbd2-f8ed-4299-9dc9-87de7a932aa9 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:58.298134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb2cbd2-f8ed-4299-9dc9-87de7a932aa9 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:58.302717 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:58.303597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:58.303914 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:58.304192 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] PATH_INFO: `/v3/users/54ff6b4fdd34405fac4ff8aa11eaa80e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:58.305386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ff6b4fdd34405fac4ff8aa11eaa80e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:58.307317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:58.352996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ff6b4fdd34405fac4ff8aa11eaa80e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.361048 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ff6b4fdd34405fac4ff8aa11eaa80e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.361740 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ff6b4fdd34405fac4ff8aa11eaa80e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.373332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ff6b4fdd34405fac4ff8aa11eaa80e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.380032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ff6b4fdd34405fac4ff8aa11eaa80e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.386272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1386/2767] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:58 2022] DELETE /identity/v3/users/54ff6b4fdd34405fac4ff8aa11eaa80e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:58.390782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:58.391078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:58.391362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:58.409269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:58.410437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:58.468299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1382/2768] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:58.477867 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:58.479257 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:58.514885 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:58.538226 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79b39bc-68ed-4abb-a281-7daa480ab80a admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:58.541926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:58.542576 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:58.542866 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:58.543148 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:58.552176 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:58.553606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:58.563910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1387/2769] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:36:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:36:58.833570 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:58.834986 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:58.847035 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1383e9-ee69-4c6c-964b-f3a796bb606b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:58.851627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:58.852525 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:58.852860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:58.853078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] PATH_INFO: `/v3/projects/4829655a00f943c48b02b08eaf91df1d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:58.854256 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4829655a00f943c48b02b08eaf91df1d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:58.856218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:58.902486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4829655a00f943c48b02b08eaf91df1d', 'request_id': 'req-73fbb9b1-c087-4a97-8550-009061f0b463'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.908694 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4829655a00f943c48b02b08eaf91df1d', 'request_id': 'req-73fbb9b1-c087-4a97-8550-009061f0b463'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.915516 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4829655a00f943c48b02b08eaf91df1d', 'request_id': 'req-73fbb9b1-c087-4a97-8550-009061f0b463'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.925653 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] The token cache is being invalidate because project 4829655a00f943c48b02b08eaf91df1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:58.925944 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.926574 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:58.928659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1383/2770] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:58 2022] DELETE /identity/v3/projects/4829655a00f943c48b02b08eaf91df1d => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:58.932601 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:58.942332 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a41fdcb-d2bb-441f-baa6-82e885714be7 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:58.997578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:58.998372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:58.998657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:58.998932 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] PATH_INFO: `/v3/users/96935360874e4d79ae45138cbca040bb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.000130 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] RBAC: Authorizing `identity:delete_user(user_id=96935360874e4d79ae45138cbca040bb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.002062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.046039 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96935360874e4d79ae45138cbca040bb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.053012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96935360874e4d79ae45138cbca040bb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.053620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96935360874e4d79ae45138cbca040bb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.065659 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96935360874e4d79ae45138cbca040bb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.073247 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-dd49fe09-8006-459a-b42e-6afbd8946e7f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96935360874e4d79ae45138cbca040bb'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.079126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1388/2771] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:36:58 2022] DELETE /identity/v3/users/96935360874e4d79ae45138cbca040bb => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:59.360079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:59.361855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:59.377107 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fbb9b1-c087-4a97-8550-009061f0b463 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:59.422848 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:59.439178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:59.440352 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.440635 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.440911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] PATH_INFO: `/v3/projects/c6cd903d52674ea1948082bda7e021a7` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.442098 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6cd903d52674ea1948082bda7e021a7)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.444045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.489074 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6cd903d52674ea1948082bda7e021a7', 'request_id': 'req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.495658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6cd903d52674ea1948082bda7e021a7', 'request_id': 'req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.502962 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6cd903d52674ea1948082bda7e021a7', 'request_id': 'req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.513553 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] The token cache is being invalidate because project c6cd903d52674ea1948082bda7e021a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:36:59.513842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.516442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-951fea35-ba6a-4ff8-a6b6-ba70a33fa3cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:36:59.518237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1384/2772] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:36:59 2022] DELETE /identity/v3/projects/c6cd903d52674ea1948082bda7e021a7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:36:59.523854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698813f-fef5-4c9f-b7e6-c912fcab4183 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.524603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698813f-fef5-4c9f-b7e6-c912fcab4183 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.524603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7698813f-fef5-4c9f-b7e6-c912fcab4183 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.543011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-7698813f-fef5-4c9f-b7e6-c912fcab4183 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:59.544201 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7698813f-fef5-4c9f-b7e6-c912fcab4183 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:59.590909 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:36:59.604279 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1389/2773] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:59.612124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149898-3116-4211-b487-6cfc5bf85ade None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.612411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149898-3116-4211-b487-6cfc5bf85ade None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.612687 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e149898-3116-4211-b487-6cfc5bf85ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.631309 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-3e149898-3116-4211-b487-6cfc5bf85ade None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:59.632542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3e149898-3116-4211-b487-6cfc5bf85ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:36:59.640391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1385/2774] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:36:59.644882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698813f-fef5-4c9f-b7e6-c912fcab4183 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:59.655401 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7698813f-fef5-4c9f-b7e6-c912fcab4183 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:59.660150 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:59.660817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.661102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.661380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.662693 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:36:59.663002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.664612 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.676882 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1390/2775] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:36:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 07 10:36:59.681482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149898-3116-4211-b487-6cfc5bf85ade None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:59.691323 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e149898-3116-4211-b487-6cfc5bf85ade None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:59.696234 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:59.696975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.697320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.697659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.698395 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.700403 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.735800 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1386/2776] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:36:59 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:59.739971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:59.750623 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd12c53-ea58-4100-8a6d-8f05077cb93f admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:59.811640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:59.812361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.812649 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.813000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.813706 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.815951 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.840658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1391/2777] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:36:59 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:36:59.842091 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:59.854307 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f7cc36-4983-4654-8c8b-86183eb98034 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:59.859504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:59.860401 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.860762 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.861111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.861904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.863560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.875544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1387/2778] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:59.878993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:59.890978 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f6b428-5296-4823-98ca-390bacd664f1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:59.897210 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0f5db-b28b-4fb6-934b-7feec69b93c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:59.898087 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f5db-b28b-4fb6-934b-7feec69b93c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.898374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f5db-b28b-4fb6-934b-7feec69b93c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.898653 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f5db-b28b-4fb6-934b-7feec69b93c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.899802 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0f5db-b28b-4fb6-934b-7feec69b93c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.901045 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b0f5db-b28b-4fb6-934b-7feec69b93c5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.912526 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1392/2779] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:36:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:36:59.916915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:36:59.932432 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c7cadf-aaa0-4fae-b01e-149e638c8da5 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:36:59.934491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:36:59.935382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.935663 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.935984 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] PATH_INFO: `/v3/projects/1b7e696f35674b87a2550a226317fac0/users/3f70cf7126d54c68b603c9ef6ecd42a1/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.956132 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b7e696f35674b87a2550a226317fac0, user_id=3f70cf7126d54c68b603c9ef6ecd42a1, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:36:59.959530 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:36:59.973347 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1388/2780] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:36:59 2022] PUT /identity/v3/projects/1b7e696f35674b87a2550a226317fac0/users/3f70cf7126d54c68b603c9ef6ecd42a1/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:36:59.978365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0388d5-9ef4-4f99-be62-d1fc4208b6a0 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:36:59.978758 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0388d5-9ef4-4f99-be62-d1fc4208b6a0 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:36:59.979183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0388d5-9ef4-4f99-be62-d1fc4208b6a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:36:59.996845 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3c0388d5-9ef4-4f99-be62-d1fc4208b6a0 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:36:59.998078 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3c0388d5-9ef4-4f99-be62-d1fc4208b6a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:00.062062 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1393/2781] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:36:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:00.073723 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:00.075485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:00.138935 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25be3ffd-2c4e-4537-b93a-098fbb0cdfc1 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:00.143458 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:00.144145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:00.144430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:00.144857 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:00.155639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:00.157274 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:00.168140 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1389/2782] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 07 10:37:00.836407 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52ed5-2c9b-453a-87f3-5148d2b70b63 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:00.836938 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52ed5-2c9b-453a-87f3-5148d2b70b63 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:00.837027 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a52ed5-2c9b-453a-87f3-5148d2b70b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:00.863993 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-06a52ed5-2c9b-453a-87f3-5148d2b70b63 None None] MFA Rules not processed for user `3f70cf7126d54c68b603c9ef6ecd42a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:00.865787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-06a52ed5-2c9b-453a-87f3-5148d2b70b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:00.914969 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:00.934422 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1394/2783] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:00.938907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:00.949019 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baaaa311-5416-47a5-a920-9f861690abdf service neutron] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:00.954667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:00.955442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:00.955781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:00.956061 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:00.956855 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:00.958860 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:00.973127 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:00.989559 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1390/2784] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:37:00 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:00.993821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a52ed5-2c9b-453a-87f3-5148d2b70b63 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:01.008000 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a52ed5-2c9b-453a-87f3-5148d2b70b63 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:01.072386 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:01.073067 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:01.073355 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:01.073637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:01.074359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:01.076317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:01.097836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1395/2785] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:37:00 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:37:01.101680 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:01.111874 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fc56a5-0d07-41c3-94f0-7428f44f54f9 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:01.116464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:01.117115 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:01.117402 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:01.117772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:01.118423 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:01.120105 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:01.130473 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1391/2786] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:01.134588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:01.144856 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9d31c0-a4ad-4d70-a7e1-ef70015b5217 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:01.150216 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:01.151090 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:01.151371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:01.151642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] PATH_INFO: `/v3/projects/fd6a9b88d3754a608787fb04d8889c71/users/31b5ff8059df4104aa13f7ec1270585b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:01.165916 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6a9b88d3754a608787fb04d8889c71, user_id=31b5ff8059df4104aa13f7ec1270585b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:01.168584 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:01.181353 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1396/2787] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:01 2022] PUT /identity/v3/projects/fd6a9b88d3754a608787fb04d8889c71/users/31b5ff8059df4104aa13f7ec1270585b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:37:01.185162 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:01.195231 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0834ab-c5d8-4df0-8c3e-a3bc36f34410 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:01.250639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5ae117-dc1d-4c9b-bd7c-a6e783ee761c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:01.251268 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ae117-dc1d-4c9b-bd7c-a6e783ee761c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:01.251590 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ae117-dc1d-4c9b-bd7c-a6e783ee761c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:01.251975 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5ae117-dc1d-4c9b-bd7c-a6e783ee761c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:01.252804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ae117-dc1d-4c9b-bd7c-a6e783ee761c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:01.254393 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5ae117-dc1d-4c9b-bd7c-a6e783ee761c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:01.264703 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1392/2788] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:01 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:01.268549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:01.278774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f06dd6e-460f-4b3f-adf0-56a7712062cf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:01.283364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:01.284351 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:01.284639 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:01.284918 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] PATH_INFO: `/v3/projects/fd6a9b88d3754a608787fb04d8889c71/users/31b5ff8059df4104aa13f7ec1270585b/roles/f5019e69971843dfadbbc6be956053fb` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:01.286899 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd6a9b88d3754a608787fb04d8889c71, user_id=31b5ff8059df4104aa13f7ec1270585b, role_id=f5019e69971843dfadbbc6be956053fb)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:01.289618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:01.299676 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1397/2789] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:01 2022] PUT /identity/v3/projects/fd6a9b88d3754a608787fb04d8889c71/users/31b5ff8059df4104aa13f7ec1270585b/roles/f5019e69971843dfadbbc6be956053fb => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:01.918486 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:01.918776 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:01.919052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:01.945116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] MFA Rules not processed for user `31b5ff8059df4104aa13f7ec1270585b`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:01.946328 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:02.015942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1393/2790] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:02.027547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:02.029313 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:02.086509 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd60c1-6a73-4fa7-876e-05ad5c5382f5 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:02.090079 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:02.090731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:02.091016 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:02.091290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:02.147560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:02.149036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:02.152549 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:02.158903 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1398/2791] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:02 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:03.498082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:03.499485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:03.524920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e991be-5b12-4270-a1a9-74f4a39dd3f2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:03.528562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:03.529220 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:03.529506 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:03.529786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:03.539772 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:03.541207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:03.551018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1394/2792] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:03 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:12.160300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:12.503237 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:12.505460 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:12.579403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464c61b-50b7-4af3-904b-b126e67c5e54 service cinder] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:12.584207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:12.585081 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:12.585365 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:12.585640 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] PATH_INFO: `/v3/users/1222d8bb50554d64ac674f109b6ad077` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:12.586881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1222d8bb50554d64ac674f109b6ad077)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:12.588922 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:12.639134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1222d8bb50554d64ac674f109b6ad077'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:12.646550 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1222d8bb50554d64ac674f109b6ad077'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:12.647212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1222d8bb50554d64ac674f109b6ad077'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:12.660497 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1222d8bb50554d64ac674f109b6ad077'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:12.669803 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-07db0d33-04fc-4c76-ada2-92887ee8b71e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1222d8bb50554d64ac674f109b6ad077'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:12.680895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1399/2793] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:12 2022] DELETE /identity/v3/users/1222d8bb50554d64ac674f109b6ad077 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:12.859576 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:12.872544 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fb30b-8e30-4880-a71e-d08c4df1cb42 service cinder] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:12.919229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:12.930271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:12.931178 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:12.931407 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:12.931683 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] PATH_INFO: `/v3/projects/1568d5b1c4bb471f9de53f4d5cf7cc82` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:12.932878 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=1568d5b1c4bb471f9de53f4d5cf7cc82)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:12.934755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:12.979277 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1568d5b1c4bb471f9de53f4d5cf7cc82', 'request_id': 'req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:12.985223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1568d5b1c4bb471f9de53f4d5cf7cc82', 'request_id': 'req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:12.992920 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1568d5b1c4bb471f9de53f4d5cf7cc82', 'request_id': 'req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:13.002821 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] The token cache is being invalidate because project 1568d5b1c4bb471f9de53f4d5cf7cc82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:13.003102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:13.003760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-6fa6b2ad-c814-4111-ae05-3eba8b71de6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:13.005367 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1395/2794] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:12 2022] DELETE /identity/v3/projects/1568d5b1c4bb471f9de53f4d5cf7cc82 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:13.010534 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a6c2-01e0-43ce-ba34-e507387a30c2 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.010940 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a6c2-01e0-43ce-ba34-e507387a30c2 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.011301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a6c2-01e0-43ce-ba34-e507387a30c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.029242 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-1aa9a6c2-01e0-43ce-ba34-e507387a30c2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:13.030556 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1aa9a6c2-01e0-43ce-ba34-e507387a30c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:13.091006 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1400/2795] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:37:13.099033 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef6df6-80b2-4601-be39-94cb7e43961e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.099356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef6df6-80b2-4601-be39-94cb7e43961e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.100190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cef6df6-80b2-4601-be39-94cb7e43961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.117781 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4cef6df6-80b2-4601-be39-94cb7e43961e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:13.118966 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4cef6df6-80b2-4601-be39-94cb7e43961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:13.126738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1396/2796] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:13.130652 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9a6c2-01e0-43ce-ba34-e507387a30c2 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:13.140004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa9a6c2-01e0-43ce-ba34-e507387a30c2 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:13.144472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:13.145120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.145403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.145679 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.146916 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:37:13.147212 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:13.148820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:13.160012 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1401/2797] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:37:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:37:13.164764 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef6df6-80b2-4601-be39-94cb7e43961e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:13.173828 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cef6df6-80b2-4601-be39-94cb7e43961e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:13.178637 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:13.179284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.179563 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.179867 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.180552 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:13.182477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:13.207213 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1397/2798] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:37:13 2022] POST /identity/v3/projects => generated 383 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:13.211633 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:13.221474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fed1a-d354-4ba0-9b24-2801056b3ea1 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:13.225572 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:13.280910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:13.281587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.281883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.282165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.282857 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:13.284804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:13.304656 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1402/2799] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:37:13 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:13.308786 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:13.319290 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249e181a-461f-4b6d-be9c-9838cbf88130 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:13.323918 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:13.324562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.324845 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.325113 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.325980 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:13.327585 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:13.338161 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1398/2800] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:13.341818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:13.352155 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9246cbb7-6274-4de6-8e61-9cfdfec03a74 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:13.356798 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5584208e-2486-4d22-b35f-e8580b2a719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:13.357477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584208e-2486-4d22-b35f-e8580b2a719b admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.357841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584208e-2486-4d22-b35f-e8580b2a719b admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.358198 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5584208e-2486-4d22-b35f-e8580b2a719b admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.359030 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584208e-2486-4d22-b35f-e8580b2a719b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:13.360832 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5584208e-2486-4d22-b35f-e8580b2a719b admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:13.371227 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1403/2801] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:13.375383 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:13.385629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201879f-a2ab-4ab9-b80c-8f99d98848b7 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:13.390133 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:13.391116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.391411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.391684 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] PATH_INFO: `/v3/projects/d81879d0f849442e9f54936c7afb26a2/users/dfe0c5a416d94ceca62f5bb1f4bdfddf/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.406557 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d81879d0f849442e9f54936c7afb26a2, user_id=dfe0c5a416d94ceca62f5bb1f4bdfddf, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:13.409202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:13.422659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1399/2802] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:13 2022] PUT /identity/v3/projects/d81879d0f849442e9f54936c7afb26a2/users/dfe0c5a416d94ceca62f5bb1f4bdfddf/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:37:13.432786 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.433023 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.433306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.460499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] MFA Rules not processed for user `dfe0c5a416d94ceca62f5bb1f4bdfddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:13.461745 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:13.527183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1404/2803] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:13.538337 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:13.540005 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:13.610704 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc13cf-c991-4549-a31b-38df21c24a19 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:13.616305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0914dada5c70437ba49688789eb31ff9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:13.616976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:13.617258 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:13.617546 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:13.625310 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:13.626899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:13.636042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1400/2804] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:16.448306 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:16.449794 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:16.515702 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bbc42c-ae8e-4870-ac1b-a77cea1ca76b None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:16.520361 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:16.521233 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:16.521519 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:16.521796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] PATH_INFO: `/v3/users/3f70cf7126d54c68b603c9ef6ecd42a1` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:16.522996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f70cf7126d54c68b603c9ef6ecd42a1)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:16.524948 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:16.540958 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:16.573071 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f70cf7126d54c68b603c9ef6ecd42a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.580478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f70cf7126d54c68b603c9ef6ecd42a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.581102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f70cf7126d54c68b603c9ef6ecd42a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.593301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f70cf7126d54c68b603c9ef6ecd42a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.600538 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f70cf7126d54c68b603c9ef6ecd42a1'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.606305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1405/2805] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:16 2022] DELETE /identity/v3/users/3f70cf7126d54c68b603c9ef6ecd42a1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:16.777528 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:16.782871 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:16.789875 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda659bc-ad1f-43ba-83be-3a4ec4f1acd8 service nova] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:16.847621 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:16.848440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:16.848720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:16.848991 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] PATH_INFO: `/v3/projects/1b7e696f35674b87a2550a226317fac0` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:16.850738 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b7e696f35674b87a2550a226317fac0)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:16.852662 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:16.896982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b7e696f35674b87a2550a226317fac0', 'request_id': 'req-02ae5233-a879-4844-bb79-3f8d3ad44d10'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.903202 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b7e696f35674b87a2550a226317fac0', 'request_id': 'req-02ae5233-a879-4844-bb79-3f8d3ad44d10'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.911021 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b7e696f35674b87a2550a226317fac0', 'request_id': 'req-02ae5233-a879-4844-bb79-3f8d3ad44d10'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.922077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] The token cache is being invalidate because project 1b7e696f35674b87a2550a226317fac0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:16.922364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.923010 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:16.924558 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1401/2806] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:16 2022] DELETE /identity/v3/projects/1b7e696f35674b87a2550a226317fac0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:16.928591 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:16.929980 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:16.943183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9f788e-7e48-4eb7-a646-d068f86a309c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:16.998651 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:16.999522 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:16.999842 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.000121 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] PATH_INFO: `/v3/users/31b5ff8059df4104aa13f7ec1270585b` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.001286 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31b5ff8059df4104aa13f7ec1270585b)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.003202 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.047159 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31b5ff8059df4104aa13f7ec1270585b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.054423 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31b5ff8059df4104aa13f7ec1270585b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.055064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31b5ff8059df4104aa13f7ec1270585b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.067597 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31b5ff8059df4104aa13f7ec1270585b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.074300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-d7b7c149-98f7-4397-921a-fd41165f8ac8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31b5ff8059df4104aa13f7ec1270585b'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.080136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1406/2807] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:16 2022] DELETE /identity/v3/users/31b5ff8059df4104aa13f7ec1270585b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:37:17.249218 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.250653 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:17.264831 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ae5233-a879-4844-bb79-3f8d3ad44d10 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:17.321366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:17.322227 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.322511 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.322785 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] PATH_INFO: `/v3/projects/fd6a9b88d3754a608787fb04d8889c71` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.324031 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd6a9b88d3754a608787fb04d8889c71)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.325956 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.364191 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:17.375348 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd6a9b88d3754a608787fb04d8889c71', 'request_id': 'req-5536fa7a-1ff9-43ff-b897-9487e8899f2b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.382070 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd6a9b88d3754a608787fb04d8889c71', 'request_id': 'req-5536fa7a-1ff9-43ff-b897-9487e8899f2b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.389332 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd6a9b88d3754a608787fb04d8889c71', 'request_id': 'req-5536fa7a-1ff9-43ff-b897-9487e8899f2b'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.400509 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] The token cache is being invalidate because project fd6a9b88d3754a608787fb04d8889c71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:17.400798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.401445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-5536fa7a-1ff9-43ff-b897-9487e8899f2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:17.402957 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1402/2808] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:17 2022] DELETE /identity/v3/projects/fd6a9b88d3754a608787fb04d8889c71 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:17.407725 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b09d0-1666-4897-909d-d0798b36723c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.408054 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b09d0-1666-4897-909d-d0798b36723c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.408363 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652b09d0-1666-4897-909d-d0798b36723c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.427687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-652b09d0-1666-4897-909d-d0798b36723c None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:17.428943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-652b09d0-1666-4897-909d-d0798b36723c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:17.437734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:DELETE=19,keystone:SELECT=79,keystone:INSERT=2 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:17.491614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1407/2809] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:17.500542 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec8e2-c433-4780-ac5a-2bb62b2719b5 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.500850 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec8e2-c433-4780-ac5a-2bb62b2719b5 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.501126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17ec8e2-c433-4780-ac5a-2bb62b2719b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.520539 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-e17ec8e2-c433-4780-ac5a-2bb62b2719b5 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:17.522381 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e17ec8e2-c433-4780-ac5a-2bb62b2719b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:17.530412 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1403/2810] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:37:17.534579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b09d0-1666-4897-909d-d0798b36723c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.544449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652b09d0-1666-4897-909d-d0798b36723c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:17.549036 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:17.549696 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.549985 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.550267 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.550996 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:37:17.551311 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.553479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.564756 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1408/2811] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:37:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:37:17.569672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec8e2-c433-4780-ac5a-2bb62b2719b5 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.578743 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17ec8e2-c433-4780-ac5a-2bb62b2719b5 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:17.583376 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:17.584050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.584334 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.584608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.585286 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.587163 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.611659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1404/2812] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:37:17 2022] POST /identity/v3/projects => generated 373 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:37:17.616115 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.625960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4308f462-73e5-4471-bd9f-bdbfd33a2348 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:17.681563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:17.682228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.682517 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.682799 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.683500 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.685437 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.704971 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1409/2813] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:37:17 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:17.709030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.719128 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748089f3-0ce9-44b9-9499-9fdedfa21e30 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:17.723677 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:17.724388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.724675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.724951 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.725664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.727305 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.737433 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1405/2814] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:17.741787 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.751402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c316fcd7-fc5c-4df8-8c97-0f3944e5c7fb admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:17.756571 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e00ca-51b2-4f78-8e3f-4a7986622ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:17.757222 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e00ca-51b2-4f78-8e3f-4a7986622ad1 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.757541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e00ca-51b2-4f78-8e3f-4a7986622ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.757808 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e00ca-51b2-4f78-8e3f-4a7986622ad1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.758532 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e00ca-51b2-4f78-8e3f-4a7986622ad1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.760166 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94e00ca-51b2-4f78-8e3f-4a7986622ad1 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.770182 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1410/2815] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:17.774370 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.784221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf45c7d3-8701-4876-9a43-3bedfe2d5ffb admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:17.788824 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:17.789690 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.789974 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.790250 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] PATH_INFO: `/v3/projects/b9e8cfef337c4d6ab8e7c8140294b0ad/users/51522900489f4be19bc9baaa60fac23e/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.805308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9e8cfef337c4d6ab8e7c8140294b0ad, user_id=51522900489f4be19bc9baaa60fac23e, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:17.808028 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:17.821200 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1406/2816] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:17 2022] PUT /identity/v3/projects/b9e8cfef337c4d6ab8e7c8140294b0ad/users/51522900489f4be19bc9baaa60fac23e/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:17.831880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64f88c-37f5-4671-a82e-330a0f955652 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:17.832171 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64f88c-37f5-4671-a82e-330a0f955652 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:17.832450 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a64f88c-37f5-4671-a82e-330a0f955652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:17.860511 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-6a64f88c-37f5-4671-a82e-330a0f955652 None None] MFA Rules not processed for user `51522900489f4be19bc9baaa60fac23e`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:17.861731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-6a64f88c-37f5-4671-a82e-330a0f955652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:17.926120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1411/2817] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:17 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:37:17.937686 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:17.939372 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:17.997854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6515f8e-2a5f-4f11-a341-65632e4e2a19 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:18.001666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bc1dcc-25e9-4819-84f6-e6e009ad3255 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:18.002308 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc1dcc-25e9-4819-84f6-e6e009ad3255 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:18.002589 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc1dcc-25e9-4819-84f6-e6e009ad3255 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:18.002864 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bc1dcc-25e9-4819-84f6-e6e009ad3255 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:18.011378 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc1dcc-25e9-4819-84f6-e6e009ad3255 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:18.012904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bc1dcc-25e9-4819-84f6-e6e009ad3255 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:18.021923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1407/2818] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:17 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:18.046935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64f88c-37f5-4671-a82e-330a0f955652 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:18.057231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:18.110011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a64f88c-37f5-4671-a82e-330a0f955652 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:18.114618 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:18.116076 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:18.116364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:18.116642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] PATH_INFO: `/v3/users/51522900489f4be19bc9baaa60fac23e` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:18.117841 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51522900489f4be19bc9baaa60fac23e)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:18.119782 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:18.163820 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51522900489f4be19bc9baaa60fac23e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.171022 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51522900489f4be19bc9baaa60fac23e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.171654 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51522900489f4be19bc9baaa60fac23e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.183997 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51522900489f4be19bc9baaa60fac23e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.190620 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51522900489f4be19bc9baaa60fac23e'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.196804 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1412/2819] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:18 2022] DELETE /identity/v3/users/51522900489f4be19bc9baaa60fac23e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:18.201281 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:18.201668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:18.201958 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:18.219034 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:18.220269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:18.282269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1408/2820] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:37:18.292634 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:18.294057 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:18.352449 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c854f7-fbda-41cf-89a7-76b6271ad448 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:18.356161 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:18.356813 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:18.357127 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:18.357408 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:18.367134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:18.368603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:18.377961 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1413/2821] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:18.657144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:18.658794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:18.668474 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:18.677042 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61235d4-63b4-4d39-850d-d52755cecfc4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:18.683697 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:18.684629 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:18.684876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:18.685154 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] PATH_INFO: `/v3/projects/b9e8cfef337c4d6ab8e7c8140294b0ad` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:18.686339 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9e8cfef337c4d6ab8e7c8140294b0ad)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:18.688482 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:18.735620 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9e8cfef337c4d6ab8e7c8140294b0ad', 'request_id': 'req-826a7402-0277-4831-b430-7c84bfd8fe59'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.741942 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9e8cfef337c4d6ab8e7c8140294b0ad', 'request_id': 'req-826a7402-0277-4831-b430-7c84bfd8fe59'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.748923 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9e8cfef337c4d6ab8e7c8140294b0ad', 'request_id': 'req-826a7402-0277-4831-b430-7c84bfd8fe59'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.760447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] The token cache is being invalidate because project b9e8cfef337c4d6ab8e7c8140294b0ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:18.760733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.761362 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-826a7402-0277-4831-b430-7c84bfd8fe59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:18.762876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1409/2822] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:18 2022] DELETE /identity/v3/projects/b9e8cfef337c4d6ab8e7c8140294b0ad => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:20.889552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:20.889552 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:20.902120 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beb6845-6369-4ae0-83cc-f9ede39a9f88 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:20.930926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:20.959730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:20.961186 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:20.961468 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:20.961741 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] PATH_INFO: `/v3/users/dfe0c5a416d94ceca62f5bb1f4bdfddf` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:20.962919 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfe0c5a416d94ceca62f5bb1f4bdfddf)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:20.964852 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.008471 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfe0c5a416d94ceca62f5bb1f4bdfddf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.015325 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfe0c5a416d94ceca62f5bb1f4bdfddf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.015974 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfe0c5a416d94ceca62f5bb1f4bdfddf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.028252 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfe0c5a416d94ceca62f5bb1f4bdfddf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.034910 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfe0c5a416d94ceca62f5bb1f4bdfddf'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.040730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1414/2823] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:20 2022] DELETE /identity/v3/users/dfe0c5a416d94ceca62f5bb1f4bdfddf => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:21.045315 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.045793 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.045894 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.063716 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:21.065419 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:21.122457 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1410/2824] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:37:21.131991 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.133895 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:21.189935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bae5e2-2403-4a17-9beb-49ccd2ae708d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.194013 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8547b3-454d-4c04-8005-523b2e46c85d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.195197 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8547b3-454d-4c04-8005-523b2e46c85d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.195498 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8547b3-454d-4c04-8005-523b2e46c85d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.195801 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8547b3-454d-4c04-8005-523b2e46c85d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.204926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8547b3-454d-4c04-8005-523b2e46c85d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.206380 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8547b3-454d-4c04-8005-523b2e46c85d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.216238 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1415/2825] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:21.479411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.481015 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:21.493803 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49852f19-d8a2-410f-b25a-648760188a32 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.498560 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.499514 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.499905 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.500246 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] PATH_INFO: `/v3/projects/d81879d0f849442e9f54936c7afb26a2` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.501500 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d81879d0f849442e9f54936c7afb26a2)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.503510 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.549899 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd81879d0f849442e9f54936c7afb26a2', 'request_id': 'req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.556608 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd81879d0f849442e9f54936c7afb26a2', 'request_id': 'req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.563727 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd81879d0f849442e9f54936c7afb26a2', 'request_id': 'req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.575144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] The token cache is being invalidate because project d81879d0f849442e9f54936c7afb26a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:21.575426 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.576052 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-d7b57c1a-0e63-4e86-a25a-6c230a1c24d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:21.577562 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1411/2826] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:21 2022] DELETE /identity/v3/projects/d81879d0f849442e9f54936c7afb26a2 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:21.582410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b72a0-ab00-449c-a208-d19ce58ab3e1 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.582703 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b72a0-ab00-449c-a208-d19ce58ab3e1 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.582983 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4b72a0-ab00-449c-a208-d19ce58ab3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.600256 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aa4b72a0-ab00-449c-a208-d19ce58ab3e1 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:21.601994 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aa4b72a0-ab00-449c-a208-d19ce58ab3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:21.648185 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:21.662151 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1416/2827] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:21.670611 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cdd9d-fe6e-48d3-b29a-03b88f40690b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.670908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cdd9d-fe6e-48d3-b29a-03b88f40690b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.671190 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5cdd9d-fe6e-48d3-b29a-03b88f40690b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.689976 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-cc5cdd9d-fe6e-48d3-b29a-03b88f40690b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:21.691183 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-cc5cdd9d-fe6e-48d3-b29a-03b88f40690b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:21.699964 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1412/2828] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:21.704303 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b72a0-ab00-449c-a208-d19ce58ab3e1 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.714672 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4b72a0-ab00-449c-a208-d19ce58ab3e1 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.719271 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.719963 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.720264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.720537 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.721266 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:37:21.721579 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.723175 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.734642 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1417/2829] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:37:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:37:21.738802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cdd9d-fe6e-48d3-b29a-03b88f40690b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.743192 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:21.749472 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5cdd9d-fe6e-48d3-b29a-03b88f40690b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.755117 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.755787 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.756073 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.756343 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.757020 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.759468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.784627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1413/2830] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:37:21 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:21.788839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.799156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41acae7-b1ea-4690-9677-dfbc8ac16535 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.854641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.855307 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.855595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.855907 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.856603 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.858525 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.877573 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1418/2831] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:37:21 2022] POST /identity/v3/users => generated 284 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:21.881627 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.892543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e4f22-a0e4-4a09-8b37-75fd37842c8c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.897195 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.897880 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.898744 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.899017 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.899770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.901391 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.911909 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1414/2832] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:21.916104 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.926510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618f38a-c739-4f51-bed6-57b5626754da admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.931143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f69886-455a-4998-83fa-8708e3814b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.931838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f69886-455a-4998-83fa-8708e3814b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.932128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f69886-455a-4998-83fa-8708e3814b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.932410 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f69886-455a-4998-83fa-8708e3814b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.933136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f69886-455a-4998-83fa-8708e3814b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.934722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f69886-455a-4998-83fa-8708e3814b7f admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.945462 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1419/2833] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:21.949504 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:21.959731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b99bc1-6793-4274-8ec9-41b62c400e8c admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:21.965494 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:21.966374 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:21.966651 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:21.966924 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] PATH_INFO: `/v3/projects/4d899e20bb994def9306651e13e4e480/users/c6e39e629d924b969b0326f6d141bb60/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:21.982447 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d899e20bb994def9306651e13e4e480, user_id=c6e39e629d924b969b0326f6d141bb60, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:21.985144 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:21.998330 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1415/2834] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:21 2022] PUT /identity/v3/projects/4d899e20bb994def9306651e13e4e480/users/c6e39e629d924b969b0326f6d141bb60/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:22.010280 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec168d7-325c-4271-8916-444dff30f66a None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:22.010580 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec168d7-325c-4271-8916-444dff30f66a None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:22.010893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec168d7-325c-4271-8916-444dff30f66a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:22.037734 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-aec168d7-325c-4271-8916-444dff30f66a None None] MFA Rules not processed for user `c6e39e629d924b969b0326f6d141bb60`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:22.039540 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-aec168d7-325c-4271-8916-444dff30f66a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:22.101995 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1420/2835] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:22.113097 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:22.115575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:22.177056 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a4f96-34b8-452b-8551-48b8feeb91f6 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:22.180681 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35edca9f-5e2a-42cd-a405-5a53313c5d9e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:22.181312 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edca9f-5e2a-42cd-a405-5a53313c5d9e service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:22.181672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edca9f-5e2a-42cd-a405-5a53313c5d9e service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:22.181945 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35edca9f-5e2a-42cd-a405-5a53313c5d9e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:22.190441 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edca9f-5e2a-42cd-a405-5a53313c5d9e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:22.192499 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35edca9f-5e2a-42cd-a405-5a53313c5d9e service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:22.202018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1416/2836] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:30.415950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec168d7-325c-4271-8916-444dff30f66a None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:30.464086 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:30.479263 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec168d7-325c-4271-8916-444dff30f66a None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:30.483865 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:30.484743 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:30.485032 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:30.485312 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] PATH_INFO: `/v3/users/c6e39e629d924b969b0326f6d141bb60` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:30.486502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6e39e629d924b969b0326f6d141bb60)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:30.488474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:30.535004 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6e39e629d924b969b0326f6d141bb60'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:30.542826 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6e39e629d924b969b0326f6d141bb60'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:30.543473 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6e39e629d924b969b0326f6d141bb60'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:30.555479 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6e39e629d924b969b0326f6d141bb60'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:30.562245 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6e39e629d924b969b0326f6d141bb60'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:30.568444 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1421/2837] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:30 2022] DELETE /identity/v3/users/c6e39e629d924b969b0326f6d141bb60 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:30.573079 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:30.573609 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:30.573714 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:30.591130 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:30.592467 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:30.649478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1417/2838] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:30.658982 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:30.660388 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:30.716617 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4682fff-4b36-4992-82d9-5963c6a7c2ea admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:30.720136 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4741ea2a-e9e2-4d2d-a3b1-4c201d2ac10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:30.720779 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741ea2a-e9e2-4d2d-a3b1-4c201d2ac10d service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:30.721064 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741ea2a-e9e2-4d2d-a3b1-4c201d2ac10d service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:30.721346 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741ea2a-e9e2-4d2d-a3b1-4c201d2ac10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:30.730774 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741ea2a-e9e2-4d2d-a3b1-4c201d2ac10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:30.732228 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4741ea2a-e9e2-4d2d-a3b1-4c201d2ac10d service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:30.741466 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1422/2839] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:31.000613 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.002048 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:31.017086 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2147f5a7-8042-4c92-a751-c55e3bf8fe33 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.022057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.023657 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.024038 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.024443 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] PATH_INFO: `/v3/projects/4d899e20bb994def9306651e13e4e480` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.026221 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d899e20bb994def9306651e13e4e480)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.028451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.071236 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:31.075229 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d899e20bb994def9306651e13e4e480', 'request_id': 'req-156e2706-a719-49c5-be72-fa1946166674'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:31.083805 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d899e20bb994def9306651e13e4e480', 'request_id': 'req-156e2706-a719-49c5-be72-fa1946166674'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:31.092520 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d899e20bb994def9306651e13e4e480', 'request_id': 'req-156e2706-a719-49c5-be72-fa1946166674'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:31.103301 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] The token cache is being invalidate because project 4d899e20bb994def9306651e13e4e480 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:31.103596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:31.104263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-156e2706-a719-49c5-be72-fa1946166674 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:31.105798 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1418/2840] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:30 2022] DELETE /identity/v3/projects/4d899e20bb994def9306651e13e4e480 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:31.111009 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205507-6271-4728-8d57-ff28d644cf77 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.111298 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205507-6271-4728-8d57-ff28d644cf77 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.111606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205507-6271-4728-8d57-ff28d644cf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.128614 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-68205507-6271-4728-8d57-ff28d644cf77 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:31.129796 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-68205507-6271-4728-8d57-ff28d644cf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:31.186733 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1423/2841] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:31.194961 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea11c6-c7e4-47ac-8df9-470d79448733 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.195254 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea11c6-c7e4-47ac-8df9-470d79448733 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.195536 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edea11c6-c7e4-47ac-8df9-470d79448733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.214223 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-edea11c6-c7e4-47ac-8df9-470d79448733 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:31.215430 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-edea11c6-c7e4-47ac-8df9-470d79448733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:31.223540 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1419/2842] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:31.228179 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205507-6271-4728-8d57-ff28d644cf77 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.237781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205507-6271-4728-8d57-ff28d644cf77 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.242315 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.242977 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.243264 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.243595 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.244305 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:37:31.244613 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.246194 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.252839 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:31.259474 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1424/2843] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:37:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:37:31.263870 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea11c6-c7e4-47ac-8df9-470d79448733 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.273729 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edea11c6-c7e4-47ac-8df9-470d79448733 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.278436 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.279083 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.279366 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.279639 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.280364 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.282263 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.307715 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1420/2844] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:37:31 2022] POST /identity/v3/projects => generated 377 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:37:31.312010 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.322424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479a2efc-b634-4245-bd95-a71fe5a8bebe admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.380219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.380927 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.381290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.381583 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.382403 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.385129 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.404812 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1425/2845] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:37:31 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:31.408897 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.419166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebce441f-d39e-47f4-864c-70acb8f0a252 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.423712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.424396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.424675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.424946 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.425655 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.427237 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.437594 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1421/2846] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:31.441499 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.452371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82165e79-c45d-45d4-a79d-f77218ed683d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.457011 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00661805-bec7-4370-8178-067b296ed95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.457674 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661805-bec7-4370-8178-067b296ed95c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.457960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661805-bec7-4370-8178-067b296ed95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.458240 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00661805-bec7-4370-8178-067b296ed95c admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.458966 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00661805-bec7-4370-8178-067b296ed95c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.460629 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00661805-bec7-4370-8178-067b296ed95c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.471372 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1426/2847] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:31.475596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.485783 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecb6d40-d42d-431a-928e-2d553b5a897a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.490420 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.491300 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.491596 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.491907 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] PATH_INFO: `/v3/projects/b21e59c4422445bf9ca21d6f9fa0511d/users/cf164607b7f84fa699c1ba25851cfa50/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.506720 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b21e59c4422445bf9ca21d6f9fa0511d, user_id=cf164607b7f84fa699c1ba25851cfa50, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.509887 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.523731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1422/2848] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:31 2022] PUT /identity/v3/projects/b21e59c4422445bf9ca21d6f9fa0511d/users/cf164607b7f84fa699c1ba25851cfa50/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:37:31.534180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ced24-c25b-4b64-bec0-ef13385abf12 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.534477 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ced24-c25b-4b64-bec0-ef13385abf12 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.534761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7ced24-c25b-4b64-bec0-ef13385abf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.560464 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-3d7ced24-c25b-4b64-bec0-ef13385abf12 None None] MFA Rules not processed for user `cf164607b7f84fa699c1ba25851cfa50`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:31.561670 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-3d7ced24-c25b-4b64-bec0-ef13385abf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:31.623448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1427/2849] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:31 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:31.635081 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:31.636876 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:31.692842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916080cd-1f29-4bac-b7fe-16669310215a admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:31.697022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5606e939-1100-41af-8418-3df0a172e538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:31.697664 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606e939-1100-41af-8418-3df0a172e538 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:31.697947 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606e939-1100-41af-8418-3df0a172e538 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:31.698225 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5606e939-1100-41af-8418-3df0a172e538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:31.706448 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606e939-1100-41af-8418-3df0a172e538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:31.707914 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5606e939-1100-41af-8418-3df0a172e538 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:31.716689 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1423/2850] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:41.620606 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:41.712676 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:49.150722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ced24-c25b-4b64-bec0-ef13385abf12 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:49.212402 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7ced24-c25b-4b64-bec0-ef13385abf12 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:49.216870 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:49.217738 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:49.218017 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:49.218289 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] PATH_INFO: `/v3/users/cf164607b7f84fa699c1ba25851cfa50` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:49.219456 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf164607b7f84fa699c1ba25851cfa50)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:49.221390 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:49.265709 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf164607b7f84fa699c1ba25851cfa50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.272943 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf164607b7f84fa699c1ba25851cfa50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.274180 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf164607b7f84fa699c1ba25851cfa50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.285942 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf164607b7f84fa699c1ba25851cfa50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.293262 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf164607b7f84fa699c1ba25851cfa50'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.299300 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1428/2851] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:49 2022] DELETE /identity/v3/users/cf164607b7f84fa699c1ba25851cfa50 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:49.303658 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:49.304143 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:49.304276 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:49.321116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:49.322314 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:49.379102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1424/2852] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:49.389299 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:49.390699 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:49.447072 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c95b8fe-913b-4605-8dbd-d81b990d0958 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:49.450744 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f3d622-bd61-443e-818e-910026c95954 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:49.451396 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3d622-bd61-443e-818e-910026c95954 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:49.451681 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3d622-bd61-443e-818e-910026c95954 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:49.451990 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f3d622-bd61-443e-818e-910026c95954 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:49.461632 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3d622-bd61-443e-818e-910026c95954 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:49.463065 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f3d622-bd61-443e-818e-910026c95954 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:49.472453 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1429/2853] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:49.736960 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:49.738409 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:49.750982 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbfa071-4b65-4c87-8850-d237a5d4734d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:49.755908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:49.756771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:49.757051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:49.757320 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] PATH_INFO: `/v3/projects/b21e59c4422445bf9ca21d6f9fa0511d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:49.759147 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b21e59c4422445bf9ca21d6f9fa0511d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:49.761089 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:49.807496 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b21e59c4422445bf9ca21d6f9fa0511d', 'request_id': 'req-f35e6149-16ec-4331-b35d-9645439a9ea6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.813755 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b21e59c4422445bf9ca21d6f9fa0511d', 'request_id': 'req-f35e6149-16ec-4331-b35d-9645439a9ea6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.822271 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b21e59c4422445bf9ca21d6f9fa0511d', 'request_id': 'req-f35e6149-16ec-4331-b35d-9645439a9ea6'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.832158 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] The token cache is being invalidate because project b21e59c4422445bf9ca21d6f9fa0511d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:49.832442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.833057 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-f35e6149-16ec-4331-b35d-9645439a9ea6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:49.834556 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1425/2854] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:49 2022] DELETE /identity/v3/projects/b21e59c4422445bf9ca21d6f9fa0511d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:49.839414 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052afc-824a-473e-9251-fb63ea151715 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:49.839698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052afc-824a-473e-9251-fb63ea151715 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:49.839999 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f052afc-824a-473e-9251-fb63ea151715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:49.856929 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-4f052afc-824a-473e-9251-fb63ea151715 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:49.858101 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-4f052afc-824a-473e-9251-fb63ea151715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:49.874752 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:49.921501 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1430/2855] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:49.929385 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a98f-9682-4962-a656-e5f6d246112e None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:49.929829 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a98f-9682-4962-a656-e5f6d246112e None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:49.930114 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4a98f-9682-4962-a656-e5f6d246112e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:49.954908 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-4ab4a98f-9682-4962-a656-e5f6d246112e None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:49.956166 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-4ab4a98f-9682-4962-a656-e5f6d246112e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:49.964685 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1426/2856] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:49.968987 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052afc-824a-473e-9251-fb63ea151715 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:49.980406 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f052afc-824a-473e-9251-fb63ea151715 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:49.985429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:49.986080 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:49.986364 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:49.986641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:49.987359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:37:49.987722 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:49.989424 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:50.001128 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1431/2857] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:37:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:37:50.005353 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4a98f-9682-4962-a656-e5f6d246112e None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:50.014491 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4a98f-9682-4962-a656-e5f6d246112e None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:50.019116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:50.019794 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:50.020082 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:50.020359 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:50.021037 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:50.022928 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:50.050078 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1427/2858] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:37:50 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:50.053545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:50.063667 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f9d86e-71ba-41ce-a5ea-d57ba73f4739 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:50.119855 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:50.120459 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:50.120746 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:50.121031 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:50.121731 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:50.123637 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:50.143485 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1432/2859] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:37:50 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:50.147524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:50.158235 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de325d-646f-47fd-b5c9-6ebbaa24e026 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:50.162788 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:50.163439 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:50.163724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:50.164051 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:50.164748 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:50.166327 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:50.176471 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1428/2860] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:50.180296 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:50.190158 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bc03d-9077-4cc5-8767-75904a432434 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:50.195830 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d52a4-86fe-4bb5-9341-01c5fcfbdce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:50.196488 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d52a4-86fe-4bb5-9341-01c5fcfbdce6 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:50.196778 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d52a4-86fe-4bb5-9341-01c5fcfbdce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:50.197056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d52a4-86fe-4bb5-9341-01c5fcfbdce6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:50.197781 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d52a4-86fe-4bb5-9341-01c5fcfbdce6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:50.199360 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d52a4-86fe-4bb5-9341-01c5fcfbdce6 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:50.209131 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1433/2861] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:50.213770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:50.224911 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a318df-8e1f-471d-8f64-629abea44478 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:50.230464 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:50.231463 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:50.231773 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:50.232045 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] PATH_INFO: `/v3/projects/7e8aaa5731e042aabfcb901f48247a6d/users/81e8e896b8504c6091b64634d1c8d486/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:50.248124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e8aaa5731e042aabfcb901f48247a6d, user_id=81e8e896b8504c6091b64634d1c8d486, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:50.250733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:50.263450 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1429/2862] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:50 2022] PUT /identity/v3/projects/7e8aaa5731e042aabfcb901f48247a6d/users/81e8e896b8504c6091b64634d1c8d486/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:50.273666 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec275a-c2e0-45d5-b91b-12920b00cc3e None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:50.273955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec275a-c2e0-45d5-b91b-12920b00cc3e None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:50.274231 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ec275a-c2e0-45d5-b91b-12920b00cc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:50.300577 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-19ec275a-c2e0-45d5-b91b-12920b00cc3e None None] MFA Rules not processed for user `81e8e896b8504c6091b64634d1c8d486`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:50.301776 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-19ec275a-c2e0-45d5-b91b-12920b00cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:50.365395 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1434/2863] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:50.376293 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:50.378060 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:50.399338 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:50.435137 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b337a1-cc9d-47d0-96b2-7ea94ea99eaf admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:50.438775 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64168224-54b9-4319-807a-a14db3611277 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:50.439416 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168224-54b9-4319-807a-a14db3611277 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:50.439698 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168224-54b9-4319-807a-a14db3611277 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:50.440003 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64168224-54b9-4319-807a-a14db3611277 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:50.448382 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64168224-54b9-4319-807a-a14db3611277 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:50.449810 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64168224-54b9-4319-807a-a14db3611277 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:50.458440 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1430/2864] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:53.402510 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec275a-c2e0-45d5-b91b-12920b00cc3e None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:53.420984 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:53.465548 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ec275a-c2e0-45d5-b91b-12920b00cc3e None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:53.470018 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:53.470893 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:53.471181 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:53.471458 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] PATH_INFO: `/v3/users/81e8e896b8504c6091b64634d1c8d486` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:53.472665 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] RBAC: Authorizing `identity:delete_user(user_id=81e8e896b8504c6091b64634d1c8d486)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:53.474578 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:53.525448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81e8e896b8504c6091b64634d1c8d486'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:53.532333 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81e8e896b8504c6091b64634d1c8d486'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:53.532950 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81e8e896b8504c6091b64634d1c8d486'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:53.545193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81e8e896b8504c6091b64634d1c8d486'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:53.552003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81e8e896b8504c6091b64634d1c8d486'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:53.557836 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1435/2865] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:37:53 2022] DELETE /identity/v3/users/81e8e896b8504c6091b64634d1c8d486 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:53.562437 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:53.562915 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:53.563022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:53.579779 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:53.580988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:53.641659 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1431/2866] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:53.652125 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:53.653658 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:53.712516 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215e67d3-6d00-4b93-8525-0fac317c392d admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:53.716173 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c49f655-bac9-4e15-b997-f93b714b8499 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:53.716824 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f655-bac9-4e15-b997-f93b714b8499 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:53.717108 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f655-bac9-4e15-b997-f93b714b8499 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:53.717385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c49f655-bac9-4e15-b997-f93b714b8499 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:53.726243 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49f655-bac9-4e15-b997-f93b714b8499 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:53.728268 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c49f655-bac9-4e15-b997-f93b714b8499 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:53.737547 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1436/2867] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:37:53.998116 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:53.999619 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:54.013730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042ccaff-1dc5-4318-a132-a929003edd4d None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.018255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.019120 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.019399 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.019672 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] PATH_INFO: `/v3/projects/7e8aaa5731e042aabfcb901f48247a6d` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.020902 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e8aaa5731e042aabfcb901f48247a6d)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.022782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.069451 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e8aaa5731e042aabfcb901f48247a6d', 'request_id': 'req-dde3748d-91d1-4f9a-a595-889b5f9d0404'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:54.075445 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e8aaa5731e042aabfcb901f48247a6d', 'request_id': 'req-dde3748d-91d1-4f9a-a595-889b5f9d0404'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:54.082739 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e8aaa5731e042aabfcb901f48247a6d', 'request_id': 'req-dde3748d-91d1-4f9a-a595-889b5f9d0404'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:54.093193 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] The token cache is being invalidate because project 7e8aaa5731e042aabfcb901f48247a6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:37:54.093470 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:54.094121 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dde3748d-91d1-4f9a-a595-889b5f9d0404 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:37:54.095643 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1432/2868] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:37:53 2022] DELETE /identity/v3/projects/7e8aaa5731e042aabfcb901f48247a6d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:54.100541 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0ad7-a748-4fd3-8a75-3851b006ffa6 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.100854 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0ad7-a748-4fd3-8a75-3851b006ffa6 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.101132 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0c0ad7-a748-4fd3-8a75-3851b006ffa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.120002 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bb0c0ad7-a748-4fd3-8a75-3851b006ffa6 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:54.121224 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bb0c0ad7-a748-4fd3-8a75-3851b006ffa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:54.153783 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:54.181291 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1437/2869] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 07 10:37:54.240724 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439294ec-7ea7-4d0d-a9d6-21df595cece4 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.241007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439294ec-7ea7-4d0d-a9d6-21df595cece4 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.241284 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439294ec-7ea7-4d0d-a9d6-21df595cece4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.259011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-439294ec-7ea7-4d0d-a9d6-21df595cece4 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:54.260232 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-439294ec-7ea7-4d0d-a9d6-21df595cece4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:54.267938 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1433/2870] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:54.272033 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c0ad7-a748-4fd3-8a75-3851b006ffa6 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:54.282269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0c0ad7-a748-4fd3-8a75-3851b006ffa6 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.286881 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.287545 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.287876 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.288160 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.288891 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:37:54.289761 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.291379 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.304588 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1438/2871] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:37:54.306804 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439294ec-7ea7-4d0d-a9d6-21df595cece4 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:54.316018 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439294ec-7ea7-4d0d-a9d6-21df595cece4 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.320884 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.321522 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.321802 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.322077 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.322751 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.324668 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.349996 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1434/2872] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:37:54 2022] POST /identity/v3/projects => generated 365 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:54.354539 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:54.364821 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1be8e4-5955-4f35-b870-89903566455c admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.420701 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.421370 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.421657 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.421934 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.422627 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.424567 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.444582 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1439/2873] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:37:54 2022] POST /identity/v3/users => generated 283 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:37:54.448733 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:54.459011 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ee10f8-3b05-4aa7-8ad7-1bfc2fe8e6fc admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.464207 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.464842 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.465123 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.465396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.466145 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.467712 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.478490 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1435/2874] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:54.482117 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:54.491960 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29711ba2-ae9d-4999-8415-8dd6bf8f56ac admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.496609 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5d1c7c-b11d-4dad-b973-b17532bf70ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.497258 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d1c7c-b11d-4dad-b973-b17532bf70ee admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.497543 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d1c7c-b11d-4dad-b973-b17532bf70ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.497817 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5d1c7c-b11d-4dad-b973-b17532bf70ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.498533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d1c7c-b11d-4dad-b973-b17532bf70ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.500780 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5d1c7c-b11d-4dad-b973-b17532bf70ee admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.510935 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1440/2875] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:37:54 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:37:54.515095 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:54.525731 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3143aa-ff09-4d20-8d83-18047a58f520 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.530322 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.531197 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.531478 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.531760 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] PATH_INFO: `/v3/projects/321e2090fdc44ef8a439388cce011381/users/f7eb5b3426df4f808edb8a837fc10dce/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.534007 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:37:54.549002 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=321e2090fdc44ef8a439388cce011381, user_id=f7eb5b3426df4f808edb8a837fc10dce, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.551667 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.563654 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1436/2876] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:37:54 2022] PUT /identity/v3/projects/321e2090fdc44ef8a439388cce011381/users/f7eb5b3426df4f808edb8a837fc10dce/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:37:54.573861 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed63b7c-ad11-4c2b-9f8d-6b539e53f7d8 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.574152 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed63b7c-ad11-4c2b-9f8d-6b539e53f7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.574429 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed63b7c-ad11-4c2b-9f8d-6b539e53f7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.600369 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-bed63b7c-ad11-4c2b-9f8d-6b539e53f7d8 None None] MFA Rules not processed for user `f7eb5b3426df4f808edb8a837fc10dce`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:37:54.601560 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-bed63b7c-ad11-4c2b-9f8d-6b539e53f7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:54.663838 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1441/2877] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:37:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:37:54.674759 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:37:54.676487 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:37:54.733417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3dc2a2-3a4a-469b-a2d0-8d6a4005a556 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:37:54.737080 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab890fc-2684-4e46-90e8-d3bf0e6aa371 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:37:54.737742 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab890fc-2684-4e46-90e8-d3bf0e6aa371 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:37:54.738022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab890fc-2684-4e46-90e8-d3bf0e6aa371 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:37:54.738294 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab890fc-2684-4e46-90e8-d3bf0e6aa371 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:37:54.746566 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab890fc-2684-4e46-90e8-d3bf0e6aa371 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:37:54.748009 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab890fc-2684-4e46-90e8-d3bf0e6aa371 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:37:54.756634 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1437/2878] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:37:54 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:38:04.660926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:04.752717 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:30.439331 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed63b7c-ad11-4c2b-9f8d-6b539e53f7d8 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:30.547469 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed63b7c-ad11-4c2b-9f8d-6b539e53f7d8 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:30.551956 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:30.552829 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:30.553111 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:30.553385 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] PATH_INFO: `/v3/users/f7eb5b3426df4f808edb8a837fc10dce` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:30.554568 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7eb5b3426df4f808edb8a837fc10dce)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:30.557207 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:30.601056 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7eb5b3426df4f808edb8a837fc10dce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:30.608348 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7eb5b3426df4f808edb8a837fc10dce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:30.608970 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7eb5b3426df4f808edb8a837fc10dce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:30.621269 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7eb5b3426df4f808edb8a837fc10dce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:30.628214 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7eb5b3426df4f808edb8a837fc10dce'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:30.634324 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1442/2879] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:38:30 2022] DELETE /identity/v3/users/f7eb5b3426df4f808edb8a837fc10dce => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:38:30.638843 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:30.639351 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:30.639465 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:30.687417 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:38:30.688631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:30.745269 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1438/2880] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:38:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:38:30.754883 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:30.756293 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:30.798493 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:30.851880 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fd63e6-23bf-4e5f-a429-04cb8948c8e7 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:30.856070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37126171-ada3-4f4f-920b-4d8042705c2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:30.856713 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126171-ada3-4f4f-920b-4d8042705c2f service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:30.856998 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126171-ada3-4f4f-920b-4d8042705c2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:30.857275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37126171-ada3-4f4f-920b-4d8042705c2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:30.866070 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37126171-ada3-4f4f-920b-4d8042705c2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:30.867504 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37126171-ada3-4f4f-920b-4d8042705c2f service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:30.876814 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1443/2881] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:38:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:38:31.141405 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.142819 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:31.154737 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1d1985-27d2-4e66-b9a9-e04ca623f035 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.159386 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.160355 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.160631 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.160912 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] PATH_INFO: `/v3/projects/321e2090fdc44ef8a439388cce011381` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.162108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=321e2090fdc44ef8a439388cce011381)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.164049 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.212126 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '321e2090fdc44ef8a439388cce011381', 'request_id': 'req-dcef90b8-de23-4b38-b321-47e1b2e5c20c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:31.218139 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '321e2090fdc44ef8a439388cce011381', 'request_id': 'req-dcef90b8-de23-4b38-b321-47e1b2e5c20c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:31.226543 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '321e2090fdc44ef8a439388cce011381', 'request_id': 'req-dcef90b8-de23-4b38-b321-47e1b2e5c20c'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:31.237388 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] The token cache is being invalidate because project 321e2090fdc44ef8a439388cce011381 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:38:31.237675 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:31.238297 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-dcef90b8-de23-4b38-b321-47e1b2e5c20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:31.239862 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1439/2882] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:38:31 2022] DELETE /identity/v3/projects/321e2090fdc44ef8a439388cce011381 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 07 10:38:31.245277 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a5c28-f000-4e3e-aee0-d5059fbf9065 None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.245569 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a5c28-f000-4e3e-aee0-d5059fbf9065 None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.245850 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a5c28-f000-4e3e-aee0-d5059fbf9065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.263337 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-ca3a5c28-f000-4e3e-aee0-d5059fbf9065 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:38:31.264619 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-ca3a5c28-f000-4e3e-aee0-d5059fbf9065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:31.322478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1444/2883] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:38:31.330988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2745689-d126-43ff-b4e6-4c8dc595a9d2 None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.331278 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2745689-d126-43ff-b4e6-4c8dc595a9d2 None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.331577 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2745689-d126-43ff-b4e6-4c8dc595a9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.349818 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-d2745689-d126-43ff-b4e6-4c8dc595a9d2 None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:38:31.351030 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-d2745689-d126-43ff-b4e6-4c8dc595a9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:31.359649 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1440/2884] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:38:31.364134 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a5c28-f000-4e3e-aee0-d5059fbf9065 None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.374682 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a5c28-f000-4e3e-aee0-d5059fbf9065 None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.379320 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.380003 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.380290 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.380566 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.381335 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80142) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 07 10:38:31.381641 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.383219 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.394862 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1445/2885] 10.222.0.16 () {58 vars in 1152 bytes} [Thu Jul 7 10:38:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 07 10:38:31.399895 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2745689-d126-43ff-b4e6-4c8dc595a9d2 None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.409151 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2745689-d126-43ff-b4e6-4c8dc595a9d2 None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.413988 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.414666 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.415442 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.415718 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.416456 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.418396 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.443138 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1441/2886] 10.222.0.16 () {60 vars in 1152 bytes} [Thu Jul 7 10:38:31 2022] POST /identity/v3/projects => generated 399 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 07 10:38:31.447433 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.457818 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36c6086-dcdf-403c-ba01-298a66e23bcf admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.514188 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.514869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.515157 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.515436 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] PATH_INFO: `/v3/users` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.516183 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.518126 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.538472 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1446/2887] 10.222.0.16 () {60 vars in 1143 bytes} [Thu Jul 7 10:38:31 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 07 10:38:31.543240 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.553524 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad524e3-7f49-4a17-9a5b-3935d37d84a0 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.558124 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.558777 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.559066 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.559342 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.560102 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.561695 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.573534 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1442/2888] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:38:31.577187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.587502 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e282e0-b134-45bd-b1f7-cfeacd34c3ef admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.592156 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e24e3c-5cd1-4724-8e14-ae7148f18931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.592815 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e24e3c-5cd1-4724-8e14-ae7148f18931 admin admin] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.593098 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e24e3c-5cd1-4724-8e14-ae7148f18931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.593374 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e24e3c-5cd1-4724-8e14-ae7148f18931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.594102 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e24e3c-5cd1-4724-8e14-ae7148f18931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.595687 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e24e3c-5cd1-4724-8e14-ae7148f18931 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.606165 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1447/2889] 10.222.0.16 () {58 vars in 1121 bytes} [Thu Jul 7 10:38:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 07 10:38:31.610196 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.620084 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691aa5e9-b534-462f-8abb-0e23f2a087a8 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.624679 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.625548 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] REQUEST_METHOD: `PUT` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.625833 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.626111 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] PATH_INFO: `/v3/projects/dca351f9555745dd829bf0a8f7a877d4/users/f8b402641270463f8cc6a4b610eee233/roles/ffd1ee8cb8334eb6a64041118960f6ee` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.640319 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dca351f9555745dd829bf0a8f7a877d4, user_id=f8b402641270463f8cc6a4b610eee233, role_id=ffd1ee8cb8334eb6a64041118960f6ee)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.642971 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.645160 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:31.657854 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1443/2890] 10.222.0.16 () {60 vars in 1482 bytes} [Thu Jul 7 10:38:31 2022] PUT /identity/v3/projects/dca351f9555745dd829bf0a8f7a877d4/users/f8b402641270463f8cc6a4b610eee233/roles/ffd1ee8cb8334eb6a64041118960f6ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:38:31.669193 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002464d8-34bc-48f7-be9d-5adface0319c None None] REQUEST_METHOD: `POST` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.669487 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002464d8-34bc-48f7-be9d-5adface0319c None None] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.669792 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002464d8-34bc-48f7-be9d-5adface0319c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.688593 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:31.697317 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.auth.core [None req-002464d8-34bc-48f7-be9d-5adface0319c None None] MFA Rules not processed for user `f8b402641270463f8cc6a4b610eee233`. Rule list: `[]` (Enabled: `True`). {{(pid=80142) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:38:31.698535 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-002464d8-34bc-48f7-be9d-5adface0319c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:31.764381 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1448/2891] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:38:31 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:38:31.776064 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:31.777761 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:31.832730 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f338d7e-91b2-4cbf-8a3c-06eed6aa7462 admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:31.836299 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edb6339-5d64-4207-b44e-7ee61367eba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc73785ed9da4ad297b924632b14e92c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:31.837489 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb6339-5d64-4207-b44e-7ee61367eba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:31.837770 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb6339-5d64-4207-b44e-7ee61367eba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:31.838044 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edb6339-5d64-4207-b44e-7ee61367eba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:31.846238 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb6339-5d64-4207-b44e-7ee61367eba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:31.847699 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edb6339-5d64-4207-b44e-7ee61367eba4 service cinder] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:31.856379 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1444/2892] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:38:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:38:38.843478 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002464d8-34bc-48f7-be9d-5adface0319c None None] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:38.906359 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002464d8-34bc-48f7-be9d-5adface0319c None None] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:38.910894 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:38.911795 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:38.912093 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:38.912371 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] PATH_INFO: `/v3/users/f8b402641270463f8cc6a4b610eee233` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:38.913562 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8b402641270463f8cc6a4b610eee233)` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:38.916088 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:38.960926 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8b402641270463f8cc6a4b610eee233'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:38.972327 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8b402641270463f8cc6a4b610eee233'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:38.972988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8b402641270463f8cc6a4b610eee233'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:38.985533 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8b402641270463f8cc6a4b610eee233'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:38.992448 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.notifications [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8b402641270463f8cc6a4b610eee233'} {{(pid=80142) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:38.998366 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1449/2893] 10.222.0.16 () {58 vars in 1223 bytes} [Thu Jul 7 10:38:38 2022] DELETE /identity/v3/users/f8b402641270463f8cc6a4b610eee233 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:38:39.002937 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] REQUEST_METHOD: `POST` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:39.003411 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:39.003541 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:39.020259 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.auth.core [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] MFA Rules not processed for user `8d3febf927af4782bc2aa5631cc4dde7`. Rule list: `[]` (Enabled: `True`). {{(pid=80141) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 07 10:38:39.021468 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:39.079578 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1445/2894] 10.222.0.16 () {58 vars in 957 bytes} [Thu Jul 7 10:38:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 07 10:38:39.089563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:39.090988 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:39.113218 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:39.152953 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f8254-c05b-4848-bafd-f68cd33c1fa8 admin admin] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:39.156625 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '61c23b3898c14a99bd1521644345d6ba', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:39.157275 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:39.157563 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:39.157843 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:39.167587 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:39.169051 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:39.178869 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1450/2895] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:38:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:38:39.443050 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:39.444593 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:39.459771 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687285a-8b12-4aa0-96b7-b48f252a375b None None] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:39.464485 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8d3febf927af4782bc2aa5631cc4dde7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3e93b5f74d6546a8b33441fdeb0e9c0d', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:39.465356 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:39.465641 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:39.465936 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] PATH_INFO: `/v3/projects/dca351f9555745dd829bf0a8f7a877d4` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:39.467171 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dca351f9555745dd829bf0a8f7a877d4)` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:39.469252 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:39.517188 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dca351f9555745dd829bf0a8f7a877d4', 'request_id': 'req-99496caf-1073-46f4-b680-1d6c9c52917f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:39.523477 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dca351f9555745dd829bf0a8f7a877d4', 'request_id': 'req-99496caf-1073-46f4-b680-1d6c9c52917f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:39.531575 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dca351f9555745dd829bf0a8f7a877d4', 'request_id': 'req-99496caf-1073-46f4-b680-1d6c9c52917f'} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:39.542614 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] The token cache is being invalidate because project dca351f9555745dd829bf0a8f7a877d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80141) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 07 10:38:39.542904 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:39.543533 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.notifications [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80141) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 07 10:38:39.545108 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1446/2896] 10.222.0.16 () {58 vars in 1232 bytes} [Thu Jul 7 10:38:39 2022] DELETE /identity/v3/projects/dca351f9555745dd829bf0a8f7a877d4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 07 10:38:49.174728 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=17 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:49.539022 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG dbcounter [-] [80141] Writing DB stats keystone:INSERT=2,keystone:SELECT=82,keystone:DELETE=7 {{(pid=80141) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:38:59.363698 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] Authenticating user token {{(pid=80142) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:38:59.365184 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:59.453272 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfa2ef5-b457-4ea8-afa7-f1f3a00b48a5 service neutron] Validating token access rules against request {{(pid=80142) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:38:59.513187 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f48754-679a-457c-8c53-91fb9373dc14 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80142) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:38:59.513851 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f48754-679a-457c-8c53-91fb9373dc14 service placement] REQUEST_METHOD: `GET` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:38:59.514143 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f48754-679a-457c-8c53-91fb9373dc14 service placement] SCRIPT_NAME: `/identity` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:38:59.514441 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f48754-679a-457c-8c53-91fb9373dc14 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80142) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:38:59.516259 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.fernet_utils [None req-15f48754-679a-457c-8c53-91fb9373dc14 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80142) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:38:59.530301 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f48754-679a-457c-8c53-91fb9373dc14 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:38:59.531955 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f48754-679a-457c-8c53-91fb9373dc14 service placement] RBAC: Authorization granted {{(pid=80142) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:38:59.542730 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: [pid: 80142|app: 0|req: 1451/2897] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:38:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 07 10:39:09.537810 dv-cinder-822821-4-iscsi devstack@keystone.service[80142]: DEBUG dbcounter [-] [80142] Writing DB stats keystone:SELECT=67 {{(pid=80142) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 07 10:44:03.348890 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Authenticating user token {{(pid=80141) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 07 10:44:03.350324 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.fernet_utils [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80141) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 07 10:44:03.364873 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99496caf-1073-46f4-b680-1d6c9c52917f admin admin] Validating token access rules against request {{(pid=80141) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 07 10:44:03.370266 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fabf7e6-549e-493f-ae0c-fa6f81a853be service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a0eae5ef8c1747408cbaad55be572987', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '3d6daaae230949659a794a503d781fb2', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80141) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 07 10:44:03.370893 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf7e6-549e-493f-ae0c-fa6f81a853be service placement] REQUEST_METHOD: `GET` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 07 10:44:03.371173 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf7e6-549e-493f-ae0c-fa6f81a853be service placement] SCRIPT_NAME: `/identity` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 07 10:44:03.371446 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fabf7e6-549e-493f-ae0c-fa6f81a853be service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80141) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 07 10:44:03.380219 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf7e6-549e-493f-ae0c-fa6f81a853be service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 07 10:44:03.381782 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fabf7e6-549e-493f-ae0c-fa6f81a853be service placement] RBAC: Authorization granted {{(pid=80141) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 07 10:44:03.392255 dv-cinder-822821-4-iscsi devstack@keystone.service[80141]: [pid: 80141|app: 0|req: 1447/2898] 10.222.0.16 () {60 vars in 1365 bytes} [Thu Jul 7 10:44:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)